[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:37 dist-test-slave-39l8 krb5kdc[7709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:37 dist-test-slave-39l8 krb5kdc[7709](info): set up 2 sockets
Aug 12 03:47:37 dist-test-slave-39l8 krb5kdc[7709](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 03:47:38.470252 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:47:38 dist-test-slave-39l8 krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:47:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:47:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:47:38.581487 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:45975
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:45975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:38.662339 7732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:38.662511 7732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:38.662532 7732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:38.663568 7732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:47:38.663606 7732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:38.663619 7732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:38.663630 7732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:47:38.663641 7732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:47:38.664984 7732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:45975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:45975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:38.665328 7732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:38.665530 7732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:38.672474 7742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:38.672492 7739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:38.672529 7732 server_base.cc:1034] running on GCE node
W20260812 03:47:38.672555 7745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:38.673029 7732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:38.673394 7732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:38.674554 7732 hybrid_clock.cc:648] HybridClock initialized: now 1786506458674532 us; error 75 us; skew 500 ppm
Aug 12 03:47:38 dist-test-slave-39l8 krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506458, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:38.677219 7732 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:47:38.677824 7732 webserver.cc:460] Webserver started at http://127.7.132.190:39927/ using document root <none> and password file <none>
I20260812 03:47:38.678189 7732 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:38.678262 7732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:38.678452 7732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:38.679641 7732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0"
format_stamp: "Formatted at 2026-08-12 03:47:38 on dist-test-slave-39l8"
server_key: "55a79d47ed6e87ead09f0e7c004415ad"
server_key_iv: "e0286e6e1ecc019728e2414eb8dadfa8"
server_key_version: "encryptionkey@0"
I20260812 03:47:38.680034 7732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0"
format_stamp: "Formatted at 2026-08-12 03:47:38 on dist-test-slave-39l8"
server_key: "55a79d47ed6e87ead09f0e7c004415ad"
server_key_iv: "e0286e6e1ecc019728e2414eb8dadfa8"
server_key_version: "encryptionkey@0"
I20260812 03:47:38.681706 7732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:47:38.682636 7757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:38.682997 7732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:38.683089 7732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0"
format_stamp: "Formatted at 2026-08-12 03:47:38 on dist-test-slave-39l8"
server_key: "55a79d47ed6e87ead09f0e7c004415ad"
server_key_iv: "e0286e6e1ecc019728e2414eb8dadfa8"
server_key_version: "encryptionkey@0"
I20260812 03:47:38.683184 7732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:38.691752 7732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:38.693476 7732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:38.693645 7732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:38.698169 7732 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:45975
I20260812 03:47:38.698223 7841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:45975 every 8 connection(s)
I20260812 03:47:38.698839 7732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:47:38.700186 7845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:38.702924 7845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0: Bootstrap starting.
I20260812 03:47:38.703740 7845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:38.704054 7845 log.cc:824] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:38.704727 7845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0: No bootstrap required, opened a new log
I20260812 03:47:38.705947 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 7732
I20260812 03:47:38.706051 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:47:38.706450 7845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45975 } }
I20260812 03:47:38.706604 7845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:38.706722 7845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2c2f1eff294f7a94bc79cd7a23d7b0, State: Initialized, Role: FOLLOWER
I20260812 03:47:38.706736 7698 external_mini_cluster.cc:1428] Setting key 7f8db76dc744adc0fab524562a6e3f87
I20260812 03:47:38.706907 7845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45975 } }
I20260812 03:47:38.707010 7845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:47:38.707048 7845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:47:38.707108 7845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:38.707598 7845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45975 } }
I20260812 03:47:38.707809 7845 leader_election.cc:302] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a2c2f1eff294f7a94bc79cd7a23d7b0; no voters:
I20260812 03:47:38.708036 7845 leader_election.cc:288] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:47:38.708068 7849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:38.708266 7849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [term 1 LEADER]: Becoming Leader. State: Replica: 8a2c2f1eff294f7a94bc79cd7a23d7b0, State: Running, Role: LEADER
I20260812 03:47:38.708405 7849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45975 } }
I20260812 03:47:38.708588 7845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:47:38.708951 7849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45975 } } }
I20260812 03:47:38.709105 7849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:38.709301 7849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a2c2f1eff294f7a94bc79cd7a23d7b0. Latest consensus state: current_term: 1 leader_uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a2c2f1eff294f7a94bc79cd7a23d7b0" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45975 } } }
I20260812 03:47:38.709434 7849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:38.709969 7860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:47:38.711114 7860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:47:38.713824 7860 catalog_manager.cc:1347] Generated new cluster ID: 1c3edf31e1bd42a5a862b023e8d19d96
I20260812 03:47:38.713876 7860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:47:38.718936 7860 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:47:38.719585 7860 catalog_manager.cc:1504] Loading token signing keys...
Aug 12 03:47:38 dist-test-slave-39l8 krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:38.723678 7860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a2c2f1eff294f7a94bc79cd7a23d7b0: Generated new TSK 0
I20260812 03:47:38.723834 7860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:47:38.725666 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:38.707913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.7.132.190:45975)
0812 03:47:38.708157 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:38.708167 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:38.708442 (+ 275us) server_negotiation.cc:407] Connection header received
0812 03:47:38.710977 (+ 2535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:38.710981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:38.711053 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:38.711215 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:38.713419 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:38.714780 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:38.716242 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:38.716778 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:38.716864 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:38.721374 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:38.721378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:38.721378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:38.721398 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:38.723521 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:38.724028 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:38.724030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:38.724031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:38.724130 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:38.724388 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:38.724389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:38.724389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:38.724678 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:38.724815 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:38.725039 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 03:47:38.725280 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:47:38.753403 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45975
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:38.853577 7886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:38.853798 7886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:38.853838 7886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:38.855053 7886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:38.855111 7886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:38.855160 7886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:47:38.856690 7886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45975
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:38.857038 7886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:38.857264 7886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:38.862329 7891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:38.862435 7894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:38.862326 7892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:38.862989 7886 server_base.cc:1034] running on GCE node
I20260812 03:47:38.863255 7886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:38.863574 7886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:38.864768 7886 hybrid_clock.cc:648] HybridClock initialized: now 1786506458864741 us; error 48 us; skew 500 ppm
Aug 12 03:47:38 dist-test-slave-39l8 krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506458, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:38.868381 7886 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:47:38.868866 7886 webserver.cc:460] Webserver started at http://127.7.132.129:34627/ using document root <none> and password file <none>
I20260812 03:47:38.869099 7886 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:38.869164 7886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:38.869323 7886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:38.870384 7886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "30a16df66eb842c98b30e58758db0f3e"
format_stamp: "Formatted at 2026-08-12 03:47:38 on dist-test-slave-39l8"
server_key: "e1e1f86cfaaf46cf632ba320742df28b"
server_key_iv: "a608e05c7c115c8d77a460cf28870d50"
server_key_version: "encryptionkey@0"
I20260812 03:47:38.870828 7886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "30a16df66eb842c98b30e58758db0f3e"
format_stamp: "Formatted at 2026-08-12 03:47:38 on dist-test-slave-39l8"
server_key: "e1e1f86cfaaf46cf632ba320742df28b"
server_key_iv: "a608e05c7c115c8d77a460cf28870d50"
server_key_version: "encryptionkey@0"
I20260812 03:47:38.872251 7886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:38.873028 7902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:38.873232 7886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:38.873324 7886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "30a16df66eb842c98b30e58758db0f3e"
format_stamp: "Formatted at 2026-08-12 03:47:38 on dist-test-slave-39l8"
server_key: "e1e1f86cfaaf46cf632ba320742df28b"
server_key_iv: "a608e05c7c115c8d77a460cf28870d50"
server_key_version: "encryptionkey@0"
I20260812 03:47:38.873409 7886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:38.881968 7886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:38.884308 7886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:38.884469 7886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:38.884759 7886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:38.885139 7886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:38.885227 7886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:38.885268 7886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:38.885311 7886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:38.892611 7886 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:39121
I20260812 03:47:38.892663 8084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:39121 every 8 connection(s)
I20260812 03:47:38.892980 7886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:47:38.898914 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 7886
I20260812 03:47:38.899013 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:47:38.899201 7698 external_mini_cluster.cc:1428] Setting key cbcbd246d0856ce54901890a5e07d8a1
Aug 12 03:47:38 dist-test-slave-39l8 krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506458, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:38.904373 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:38.894053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:51423 (local address 127.7.132.190:45975)
0812 03:47:38.894125 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:38.894128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:38.894370 (+ 242us) server_negotiation.cc:407] Connection header received
0812 03:47:38.895032 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:38.895033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:38.895043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:38.895097 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:38.896354 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:38.896829 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:38.898198 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:38.898375 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:38.898424 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:38.901342 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:38.901343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:38.901344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:38.901351 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:38.902970 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:38.903432 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:38.903433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:38.903434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:38.903497 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:38.903753 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:38.903754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:38.903754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:38.903861 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:38.903928 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:38.904195 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 03:47:38.904291 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:47:38.904518 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:38.894036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45975 (local address 127.7.132.129:51423)
0812 03:47:38.894233 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:38.894278 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:38.894939 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:38.895163 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:38.895167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:38.896021 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:38.896268 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:38.896274 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:38.896912 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:38.896913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:38.898119 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:38.898121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:38.898449 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:38.898450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:38.898588 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:38.899262 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:38.899274 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:38.901216 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:38.903074 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:38.903082 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:38.903084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:38.903355 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:38.903571 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:38.903571 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:38.903572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:38.903679 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:38.903932 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:38.903939 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:38.904065 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 03:47:38.904158 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 03:47:38.904308 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260812 03:47:38.905126 8085 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45975
I20260812 03:47:38.905248 8085 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:38.905452 8085 heartbeater.cc:511] Master 127.7.132.190:45975 requested a full tablet report, sending...
I20260812 03:47:38.906314 7782 ts_manager.cc:194] Registered new tserver with Master: 30a16df66eb842c98b30e58758db0f3e (127.7.132.129:39121)
I20260812 03:47:38.907104 7782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:51423
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:47:38.925454 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45975
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:39.007359 8093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:39.007555 8093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:39.007578 8093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:39.008703 8093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:39.008744 8093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:39.008770 8093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:47:39.010317 8093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45975
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:39.010535 8093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:39.010782 8093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:39.015623 8107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:39.015642 8100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:39.015642 8099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:39.015995 8093 server_base.cc:1034] running on GCE node
I20260812 03:47:39.016182 8093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:39.016412 8093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:39.017570 8093 hybrid_clock.cc:648] HybridClock initialized: now 1786506459017550 us; error 30 us; skew 500 ppm
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:39.020097 8093 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:47:39.020457 8093 webserver.cc:460] Webserver started at http://127.7.132.130:35505/ using document root <none> and password file <none>
I20260812 03:47:39.020638 8093 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:39.020677 8093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:39.020772 8093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:39.021751 8093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "4dcb3badd83a44a2aa41d3ead1b2c7cd"
format_stamp: "Formatted at 2026-08-12 03:47:39 on dist-test-slave-39l8"
server_key: "fc4352adf33ae0955b705fc5fc53d66d"
server_key_iv: "aa1698f02ef2fc34504d2bb31bafe645"
server_key_version: "encryptionkey@0"
I20260812 03:47:39.022063 8093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "4dcb3badd83a44a2aa41d3ead1b2c7cd"
format_stamp: "Formatted at 2026-08-12 03:47:39 on dist-test-slave-39l8"
server_key: "fc4352adf33ae0955b705fc5fc53d66d"
server_key_iv: "aa1698f02ef2fc34504d2bb31bafe645"
server_key_version: "encryptionkey@0"
I20260812 03:47:39.023334 8093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 03:47:39.024003 8117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:39.024168 8093 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:39.024253 8093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "4dcb3badd83a44a2aa41d3ead1b2c7cd"
format_stamp: "Formatted at 2026-08-12 03:47:39 on dist-test-slave-39l8"
server_key: "fc4352adf33ae0955b705fc5fc53d66d"
server_key_iv: "aa1698f02ef2fc34504d2bb31bafe645"
server_key_version: "encryptionkey@0"
I20260812 03:47:39.024307 8093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:39.055472 8093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:39.059156 8093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:39.059345 8093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:39.060575 8093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:39.060885 8093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:39.060935 8093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:39.060976 8093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:39.061000 8093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:39.068307 8093 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37491
I20260812 03:47:39.068347 8306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37491 every 8 connection(s)
I20260812 03:47:39.068801 8093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:47:39.070847 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8093
I20260812 03:47:39.070952 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:47:39.071123 7698 external_mini_cluster.cc:1428] Setting key d6697887d910cabf715a75efd679fc47
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
I20260812 03:47:39.081744 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:39.069725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:35283 (local address 127.7.132.190:45975)
0812 03:47:39.069788 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:39.069792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:39.070061 (+ 269us) server_negotiation.cc:407] Connection header received
0812 03:47:39.070962 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:39.070964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:39.070984 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:39.071040 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:39.072822 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.073620 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.075186 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.075617 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.075759 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:39.078542 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:39.078544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:39.078544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:39.078552 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:39.080323 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:39.080738 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:39.080740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:39.080740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:39.080799 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:39.081072 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:39.081073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:39.081073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:39.081221 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:39.081284 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:39.081560 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 03:47:39.081644 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:47:39.081921 8313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:39.069708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45975 (local address 127.7.132.130:35283)
0812 03:47:39.069925 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:39.069975 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:39.070819 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:39.071156 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:39.071161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:39.072370 (+ 1209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:39.072676 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:39.072683 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.073753 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.073754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:39.075075 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:39.075079 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.075861 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.075862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:39.076127 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:39.076814 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:39.076824 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:39.078453 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:39.080420 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:39.080428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:39.080430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:39.080649 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:39.080851 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:39.080852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:39.080853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:39.080997 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:39.081275 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:39.081281 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:39.081419 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 03:47:39.081522 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 03:47:39.081679 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260812 03:47:39.082530 8307 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45975
I20260812 03:47:39.082638 8307 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:39.082896 8307 heartbeater.cc:511] Master 127.7.132.190:45975 requested a full tablet report, sending...
I20260812 03:47:39.083335 7782 ts_manager.cc:194] Registered new tserver with Master: 4dcb3badd83a44a2aa41d3ead1b2c7cd (127.7.132.130:37491)
I20260812 03:47:39.083868 7782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:35283
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:47:39.098894 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45975
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:39.179667 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:39.179909 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:39.179949 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:39.181047 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:39.181108 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:39.181154 8317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:47:39.182660 8317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45975
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:39.182941 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:39.183152 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:39.187994 8326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:39.188165 8317 server_base.cc:1034] running on GCE node
W20260812 03:47:39.188019 8333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:39.188037 8328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:39.188460 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:39.188737 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:39.189877 8317 hybrid_clock.cc:648] HybridClock initialized: now 1786506459189865 us; error 27 us; skew 500 ppm
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:39.192407 8317 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:47:39.192780 8317 webserver.cc:460] Webserver started at http://127.7.132.131:40727/ using document root <none> and password file <none>
I20260812 03:47:39.193017 8317 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:39.193061 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:39.193147 8317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:39.194088 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "9612d3a2c6a04781af6500bb15667c7d"
format_stamp: "Formatted at 2026-08-12 03:47:39 on dist-test-slave-39l8"
server_key: "1c7954d19a5e1a2551d9387f9d182768"
server_key_iv: "4e8701cbff3921dd2d36bf5d00c3b1b5"
server_key_version: "encryptionkey@0"
I20260812 03:47:39.194394 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "9612d3a2c6a04781af6500bb15667c7d"
format_stamp: "Formatted at 2026-08-12 03:47:39 on dist-test-slave-39l8"
server_key: "1c7954d19a5e1a2551d9387f9d182768"
server_key_iv: "4e8701cbff3921dd2d36bf5d00c3b1b5"
server_key_version: "encryptionkey@0"
I20260812 03:47:39.195689 8317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:47:39.196309 8341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:39.196468 8317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:39.196568 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "9612d3a2c6a04781af6500bb15667c7d"
format_stamp: "Formatted at 2026-08-12 03:47:39 on dist-test-slave-39l8"
server_key: "1c7954d19a5e1a2551d9387f9d182768"
server_key_iv: "4e8701cbff3921dd2d36bf5d00c3b1b5"
server_key_version: "encryptionkey@0"
I20260812 03:47:39.196636 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:39.206929 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:39.209201 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:39.209347 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:39.209580 8317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:39.209928 8317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:39.209988 8317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:39.210027 8317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:39.210053 8317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:39.216317 8317 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:40127
I20260812 03:47:39.216383 8525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:40127 every 8 connection(s)
I20260812 03:47:39.216677 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506459, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:39.223356 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8317
I20260812 03:47:39.223459 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:47:39.223717 7698 external_mini_cluster.cc:1428] Setting key 36537efbb074300f7bf31255b7320d42
I20260812 03:47:39.225666 8531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:39.217397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45975 (local address 127.7.132.131:37797)
0812 03:47:39.217553 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:39.217592 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:39.218039 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:39.218244 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:39.218247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:39.218859 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:39.219040 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:39.219045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.219664 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.219664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:39.220513 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:39.220515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.220852 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.220853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:39.220942 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:39.221308 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:39.221315 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:39.222615 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:39.224372 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:39.224379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:39.224380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:39.224701 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:39.224902 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:39.224903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:39.224903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:39.225027 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:39.225249 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:39.225252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:39.225346 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 03:47:39.225420 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:47:39.225526 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260812 03:47:39.225662 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:39.217427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37797 (local address 127.7.132.190:45975)
0812 03:47:39.217488 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:39.217490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:39.217674 (+ 184us) server_negotiation.cc:407] Connection header received
0812 03:47:39.218121 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:39.218122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:39.218132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:39.218186 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:39.219116 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.219588 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.220584 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.220785 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.220836 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:39.222745 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:39.222747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:39.222747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:39.222755 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:39.224305 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:39.224774 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:39.224775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:39.224776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:39.224843 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:39.225090 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:39.225103 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:39.225103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:39.225206 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:39.225260 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:39.225466 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 03:47:39.225556 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:47:39.226078 8526 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45975
I20260812 03:47:39.226158 8526 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:39.226293 8526 heartbeater.cc:511] Master 127.7.132.190:45975 requested a full tablet report, sending...
I20260812 03:47:39.226613 7782 ts_manager.cc:194] Registered new tserver with Master: 9612d3a2c6a04781af6500bb15667c7d (127.7.132.131:40127)
I20260812 03:47:39.227005 7782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:37797
I20260812 03:47:39.235644 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506459, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506459, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:47:39.258505 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:39.251602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34404 (local address 127.7.132.129:39121)
0812 03:47:39.251755 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:39.251759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:39.251769 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:47:39.251793 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:39.251795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:39.251807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:39.251867 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:39.252396 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.253065 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.253747 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.254025 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.254087 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:39.256166 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:39.256174 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:39.256174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:39.256181 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:39.257513 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:39.257820 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:39.257823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:39.257823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:39.257888 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:39.258041 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:39.258042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:39.258042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:39.258158 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:39.258252 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:39.258310 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 03:47:39.258368 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
W20260812 03:47:39.258980 7950 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34404
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[7709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[7709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:47:39.280058 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:39.273660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.7.132.129:39121)
0812 03:47:39.273718 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:39.273720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:39.273786 (+ 66us) server_negotiation.cc:407] Connection header received
0812 03:47:39.273859 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:39.273860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:39.273875 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:39.273927 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:39.274407 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.274897 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.275604 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:39.275812 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:39.275863 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:39.278012 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:39.278013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:39.278014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:39.278021 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:39.279172 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:39.279478 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:39.279479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:39.279479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:39.279527 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:39.279675 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:39.279675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:39.279675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:39.279754 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:39.279873 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:39.279935 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 03:47:39.280003 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:47:39.280475 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 7886
I20260812 03:47:39.296005 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8093
I20260812 03:47:39.320847 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8317
I20260812 03:47:39.329005 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 7732
2026-08-12T03:47:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2076 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[8555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[8555](info): set up 2 sockets
Aug 12 03:47:39 dist-test-slave-39l8 krb5kdc[8555](info): commencing operation
krb5kdc: starting...
W20260812 03:47:40.527127 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:47:40 dist-test-slave-39l8 krb5kdc[8555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:47:40.639056 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:35599
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:35599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:40.717734 8577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:40.717916 8577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:40.717936 8577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:40.719030 8577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:47:40.719110 8577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:40.719125 8577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:40.719136 8577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:47:40.719149 8577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:47:40.720618 8577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:35599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:35599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:40.720860 8577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:40.721084 8577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:40.726018 8590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:40.726028 8587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:40.726017 8588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:40.726346 8577 server_base.cc:1034] running on GCE node
I20260812 03:47:40.726553 8577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:40.726840 8577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:40.727977 8577 hybrid_clock.cc:648] HybridClock initialized: now 1786506460727963 us; error 29 us; skew 500 ppm
Aug 12 03:47:40 dist-test-slave-39l8 krb5kdc[8555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506460, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:40.730448 8577 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:47:40.730938 8577 webserver.cc:460] Webserver started at http://127.7.132.190:34287/ using document root <none> and password file <none>
I20260812 03:47:40.731189 8577 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:40.731256 8577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:40.731396 8577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:40.732450 8577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "dd134fd31ea84bb4a99a95f51b30d76a"
format_stamp: "Formatted at 2026-08-12 03:47:40 on dist-test-slave-39l8"
server_key: "4a5ade7f0ceea846de638bd635dc25c5"
server_key_iv: "ce911c366b712f7dd7d12418db4db41b"
server_key_version: "encryptionkey@0"
I20260812 03:47:40.732795 8577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "dd134fd31ea84bb4a99a95f51b30d76a"
format_stamp: "Formatted at 2026-08-12 03:47:40 on dist-test-slave-39l8"
server_key: "4a5ade7f0ceea846de638bd635dc25c5"
server_key_iv: "ce911c366b712f7dd7d12418db4db41b"
server_key_version: "encryptionkey@0"
I20260812 03:47:40.734095 8577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:47:40.734794 8602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:40.734972 8577 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:40.735046 8577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "dd134fd31ea84bb4a99a95f51b30d76a"
format_stamp: "Formatted at 2026-08-12 03:47:40 on dist-test-slave-39l8"
server_key: "4a5ade7f0ceea846de638bd635dc25c5"
server_key_iv: "ce911c366b712f7dd7d12418db4db41b"
server_key_version: "encryptionkey@0"
I20260812 03:47:40.735114 8577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:40.751039 8577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:40.752696 8577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:40.752799 8577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:40.756718 8577 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:35599
I20260812 03:47:40.756767 8690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:35599 every 8 connection(s)
I20260812 03:47:40.757064 8577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:47:40.757835 8693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:40.760116 8693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a: Bootstrap starting.
I20260812 03:47:40.760685 8693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:40.760969 8693 log.cc:824] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:40.761613 8693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a: No bootstrap required, opened a new log
I20260812 03:47:40.763175 8693 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd134fd31ea84bb4a99a95f51b30d76a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35599 } }
I20260812 03:47:40.763350 8693 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:40.763353 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8577
I20260812 03:47:40.763398 8693 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd134fd31ea84bb4a99a95f51b30d76a, State: Initialized, Role: FOLLOWER
I20260812 03:47:40.763437 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:47:40.763579 7698 external_mini_cluster.cc:1428] Setting key 6070f45526c4826cf449a1fc1ff60fef
I20260812 03:47:40.763573 8693 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd134fd31ea84bb4a99a95f51b30d76a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35599 } }
I20260812 03:47:40.763675 8693 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:47:40.763713 8693 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:47:40.763779 8693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:40.765014 8693 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd134fd31ea84bb4a99a95f51b30d76a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35599 } }
I20260812 03:47:40.765132 8693 leader_election.cc:302] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd134fd31ea84bb4a99a95f51b30d76a; no voters:
I20260812 03:47:40.765305 8693 leader_election.cc:288] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:47:40.765344 8703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:40.765476 8703 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [term 1 LEADER]: Becoming Leader. State: Replica: dd134fd31ea84bb4a99a95f51b30d76a, State: Running, Role: LEADER
I20260812 03:47:40.765555 8693 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:47:40.765560 8703 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd134fd31ea84bb4a99a95f51b30d76a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35599 } }
I20260812 03:47:40.765928 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd134fd31ea84bb4a99a95f51b30d76a. Latest consensus state: current_term: 1 leader_uuid: "dd134fd31ea84bb4a99a95f51b30d76a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd134fd31ea84bb4a99a95f51b30d76a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35599 } } }
I20260812 03:47:40.766010 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:40.765944 8704 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd134fd31ea84bb4a99a95f51b30d76a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd134fd31ea84bb4a99a95f51b30d76a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 35599 } } }
I20260812 03:47:40.766182 8704 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:40.766286 8716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:47:40.766765 8716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:47:40.768273 8716 catalog_manager.cc:1347] Generated new cluster ID: a05fb8aa9e32432882844b33a5eeaf94
I20260812 03:47:40.768347 8716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:47:40 dist-test-slave-39l8 krb5kdc[8555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:40.774369 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:40.764076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.7.132.190:35599)
0812 03:47:40.764263 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:40.764270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:40.764294 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:47:40.764392 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:40.764397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:40.764434 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:40.764535 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:40.765353 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:40.766720 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:40.767758 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:40.768285 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:40.768411 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:40.770611 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:40.770614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:40.770615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:40.770626 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:40.772323 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:40.772692 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:40.772695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:40.772695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:40.772794 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:40.773007 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:40.773008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:40.773009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:40.773254 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:40.774082 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:40.774137 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:47:40.774241 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260812 03:47:40.792286 8716 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:47:40.792959 8716 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:47:40.797257 8716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd134fd31ea84bb4a99a95f51b30d76a: Generated new TSK 0
I20260812 03:47:40.797406 8716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:47:40.823660 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35599
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:40.903096 8741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:40.903291 8741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:40.903314 8741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:40.904341 8741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:40.904373 8741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:40.904399 8741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:47:40.905827 8741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35599
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:40.905992 8741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:40.906142 8741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:40.910935 8756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:40.910949 8753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:40.911054 8741 server_base.cc:1034] running on GCE node
W20260812 03:47:40.911109 8754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:40.911288 8741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:40.911520 8741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:40.912673 8741 hybrid_clock.cc:648] HybridClock initialized: now 1786506460912660 us; error 43 us; skew 500 ppm
Aug 12 03:47:40 dist-test-slave-39l8 krb5kdc[8555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506460, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:40.915002 8741 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:47:40.915408 8741 webserver.cc:460] Webserver started at http://127.7.132.129:43467/ using document root <none> and password file <none>
I20260812 03:47:40.915611 8741 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:40.915653 8741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:40.915786 8741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:40.916806 8741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "ae10af3cfb494127bf96ed820154b7ea"
format_stamp: "Formatted at 2026-08-12 03:47:40 on dist-test-slave-39l8"
server_key: "f96244d33b3c057ab0af6ced8c1323f7"
server_key_iv: "b649814813ab66518381c097542cfa50"
server_key_version: "encryptionkey@0"
I20260812 03:47:40.917152 8741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "ae10af3cfb494127bf96ed820154b7ea"
format_stamp: "Formatted at 2026-08-12 03:47:40 on dist-test-slave-39l8"
server_key: "f96244d33b3c057ab0af6ced8c1323f7"
server_key_iv: "b649814813ab66518381c097542cfa50"
server_key_version: "encryptionkey@0"
I20260812 03:47:40.918481 8741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:47:40.919248 8766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:40.919456 8741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:40.919543 8741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "ae10af3cfb494127bf96ed820154b7ea"
format_stamp: "Formatted at 2026-08-12 03:47:40 on dist-test-slave-39l8"
server_key: "f96244d33b3c057ab0af6ced8c1323f7"
server_key_iv: "b649814813ab66518381c097542cfa50"
server_key_version: "encryptionkey@0"
I20260812 03:47:40.919615 8741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:40.931564 8741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:40.933681 8741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:40.933790 8741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:40.933955 8741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:40.934247 8741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:40.934280 8741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:40.934322 8741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:40.934350 8741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:40.940047 8741 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:38837
I20260812 03:47:40.940217 8959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:38837 every 8 connection(s)
I20260812 03:47:40.940418 8741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:47:40 dist-test-slave-39l8 krb5kdc[8555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506460, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:40.947986 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8741
I20260812 03:47:40.948062 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:47:40.948244 7698 external_mini_cluster.cc:1428] Setting key d3486ef911162f509a8546c7a63909dd
I20260812 03:47:40.950253 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:40.941136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:59067 (local address 127.7.132.190:35599)
0812 03:47:40.941185 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:40.941187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:40.941366 (+ 179us) server_negotiation.cc:407] Connection header received
0812 03:47:40.942552 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:40.942553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:40.942568 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:40.942628 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:40.943262 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:40.943723 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:40.944649 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:40.944839 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:40.944891 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:40.947332 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:40.947333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:40.947333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:40.947340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:40.948836 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:40.949339 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:40.949339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:40.949340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:40.949391 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:40.949661 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:40.949661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:40.949662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:40.949757 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:40.949808 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:40.950124 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 03:47:40.950192 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:47:40.950527 8962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:40.941100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:35599 (local address 127.7.132.129:59067)
0812 03:47:40.941260 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:40.941299 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:40.942360 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:40.942695 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:40.942698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:40.942990 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:40.943181 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:40.943185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:40.943795 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:40.943795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:40.944575 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:40.944577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:40.944911 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:40.944912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:40.945006 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:40.945502 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:40.945510 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:40.947194 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:40.948917 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:40.948936 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:40.948939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:40.949268 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:40.949456 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:40.949456 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:40.949457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:40.949600 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:40.949827 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:40.949834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:40.949994 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 03:47:40.950102 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:47:40.950277 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260812 03:47:40.951103 8960 heartbeater.cc:347] Connected to a master server at 127.7.132.190:35599
I20260812 03:47:40.951203 8960 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:40.951382 8960 heartbeater.cc:511] Master 127.7.132.190:35599 requested a full tablet report, sending...
I20260812 03:47:40.951857 8631 ts_manager.cc:194] Registered new tserver with Master: ae10af3cfb494127bf96ed820154b7ea (127.7.132.129:38837)
I20260812 03:47:40.952494 8631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:59067
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:47:40.973089 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35599
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:41.059665 8970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:41.059876 8970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:41.059900 8970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:41.061033 8970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:41.061074 8970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:41.061100 8970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:47:41.062568 8970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35599
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:41.062832 8970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:41.063053 8970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:41.067932 8976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:41.067932 8975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:41.067958 8984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:41.068274 8970 server_base.cc:1034] running on GCE node
I20260812 03:47:41.068485 8970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:41.068701 8970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:41.069851 8970 hybrid_clock.cc:648] HybridClock initialized: now 1786506461069834 us; error 31 us; skew 500 ppm
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506461, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:41.075240 8970 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:47:41.075793 8970 webserver.cc:460] Webserver started at http://127.7.132.130:40377/ using document root <none> and password file <none>
I20260812 03:47:41.076081 8970 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:41.076151 8970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:41.076294 8970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:41.077312 8970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "1c92337487c44c3788f630c290f94fe4"
format_stamp: "Formatted at 2026-08-12 03:47:41 on dist-test-slave-39l8"
server_key: "acf8be5ac21b09275cbaa119b92e7be6"
server_key_iv: "470256cd77616d7d7aedfa7e4249afe4"
server_key_version: "encryptionkey@0"
I20260812 03:47:41.077606 8970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "1c92337487c44c3788f630c290f94fe4"
format_stamp: "Formatted at 2026-08-12 03:47:41 on dist-test-slave-39l8"
server_key: "acf8be5ac21b09275cbaa119b92e7be6"
server_key_iv: "470256cd77616d7d7aedfa7e4249afe4"
server_key_version: "encryptionkey@0"
I20260812 03:47:41.078895 8970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:47:41.079550 8998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:41.079710 8970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:41.079802 8970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "1c92337487c44c3788f630c290f94fe4"
format_stamp: "Formatted at 2026-08-12 03:47:41 on dist-test-slave-39l8"
server_key: "acf8be5ac21b09275cbaa119b92e7be6"
server_key_iv: "470256cd77616d7d7aedfa7e4249afe4"
server_key_version: "encryptionkey@0"
I20260812 03:47:41.079859 8970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:41.102761 8970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:41.104848 8970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:41.104961 8970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:41.105099 8970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:41.105424 8970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:41.105456 8970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:41.105504 8970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:41.105535 8970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:41.111796 8970 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:33209
I20260812 03:47:41.111907 9215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:33209 every 8 connection(s)
I20260812 03:47:41.112241 8970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506461, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:41.121055 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.112985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:33233 (local address 127.7.132.190:35599)
0812 03:47:41.113034 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.113036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.113293 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:47:41.114189 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.114190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.114203 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.114263 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:41.114939 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.115427 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.116342 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.116560 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.116640 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.118448 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:41.118450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:41.118450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:41.118458 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:41.119683 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.120074 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.120076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.120076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.120131 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.120376 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.120377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.120377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.120482 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:41.120551 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.120865 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.120963 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:47:41.121189 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.112936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:35599 (local address 127.7.132.130:33233)
0812 03:47:41.113174 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:41.113215 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:41.114080 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:41.114328 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:41.114331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:41.114585 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:41.114841 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.114845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.115513 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.115514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.116258 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.116260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.116652 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.116653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.116789 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.117158 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:41.117166 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:41.118363 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:41.119766 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:41.119773 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:41.119775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:41.119987 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:41.120200 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:41.120201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:41.120201 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:41.120306 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:41.120552 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:41.120559 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:41.120698 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 03:47:41.120828 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 03:47:41.120983 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
I20260812 03:47:41.121618 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8970
I20260812 03:47:41.121712 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:47:41.121754 9216 heartbeater.cc:347] Connected to a master server at 127.7.132.190:35599
I20260812 03:47:41.121846 9216 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:41.121893 7698 external_mini_cluster.cc:1428] Setting key 86d29470e831230d76908b33930451cc
I20260812 03:47:41.122061 9216 heartbeater.cc:511] Master 127.7.132.190:35599 requested a full tablet report, sending...
I20260812 03:47:41.122457 8629 ts_manager.cc:194] Registered new tserver with Master: 1c92337487c44c3788f630c290f94fe4 (127.7.132.130:33209)
I20260812 03:47:41.122964 8629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:33233
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:47:41.144827 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35599
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:41.226742 9225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:41.226960 9225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:41.226999 9225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:41.228137 9225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:41.228199 9225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:41.228245 9225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:47:41.229735 9225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:35599
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:41.229986 9225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:41.230217 9225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:41.235162 9236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:41.235170 9231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:41.235216 9233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:41.235282 9225 server_base.cc:1034] running on GCE node
I20260812 03:47:41.235621 9225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:41.235877 9225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:41.237037 9225 hybrid_clock.cc:648] HybridClock initialized: now 1786506461237030 us; error 31 us; skew 500 ppm
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506461, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:41.239686 9225 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:47:41.240084 9225 webserver.cc:460] Webserver started at http://127.7.132.131:38365/ using document root <none> and password file <none>
I20260812 03:47:41.240257 9225 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:41.240296 9225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:41.240392 9225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:41.241376 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "6466e42574ad4e85b1dadbdf11e72da9"
format_stamp: "Formatted at 2026-08-12 03:47:41 on dist-test-slave-39l8"
server_key: "6816495d05fe96a398c2fc5ebc3307d4"
server_key_iv: "08bc24096c4eb12d3021515311bde95e"
server_key_version: "encryptionkey@0"
I20260812 03:47:41.241707 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "6466e42574ad4e85b1dadbdf11e72da9"
format_stamp: "Formatted at 2026-08-12 03:47:41 on dist-test-slave-39l8"
server_key: "6816495d05fe96a398c2fc5ebc3307d4"
server_key_iv: "08bc24096c4eb12d3021515311bde95e"
server_key_version: "encryptionkey@0"
I20260812 03:47:41.242969 9225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:47:41.243665 9247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:41.243834 9225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:41.243940 9225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "6466e42574ad4e85b1dadbdf11e72da9"
format_stamp: "Formatted at 2026-08-12 03:47:41 on dist-test-slave-39l8"
server_key: "6816495d05fe96a398c2fc5ebc3307d4"
server_key_iv: "08bc24096c4eb12d3021515311bde95e"
server_key_version: "encryptionkey@0"
I20260812 03:47:41.244015 9225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:41.267926 9225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:41.270089 9225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:41.270247 9225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:41.270466 9225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:41.270836 9225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:41.270885 9225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:41.270924 9225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:41.270951 9225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:41.276793 9225 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:40809
I20260812 03:47:41.276837 9431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:40809 every 8 connection(s)
I20260812 03:47:41.277135 9225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:47:41.279361 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9225
I20260812 03:47:41.279443 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:47:41.279598 7698 external_mini_cluster.cc:1428] Setting key 423c63772fd4bc89b2e8d67496192dfe
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506461, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:41.286324 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.277748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:56367 (local address 127.7.132.190:35599)
0812 03:47:41.277799 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.277802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.278049 (+ 247us) server_negotiation.cc:407] Connection header received
0812 03:47:41.279061 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.279062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.279075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.279142 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:41.279877 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.280705 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.281704 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.281899 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.281979 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.284066 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:41.284067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:41.284067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:41.284074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:41.285238 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.285577 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.285578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.285578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.285626 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.285815 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.285815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.285816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.285904 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:41.285950 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.286173 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.286259 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:47:41.286505 9434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.277736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:35599 (local address 127.7.132.131:56367)
0812 03:47:41.277929 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:41.277973 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:41.278959 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:41.279210 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:41.279235 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:41.279533 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:41.279788 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.279795 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.280799 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.280800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.281614 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.281616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.281991 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.281991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.282114 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.282564 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:41.282572 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:41.283951 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:41.285319 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:41.285325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:41.285327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:41.285519 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:41.285680 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:41.285681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:41.285681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:41.285768 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:41.285952 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:41.285955 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:41.286052 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:47:41.286144 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:47:41.286309 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260812 03:47:41.286957 9432 heartbeater.cc:347] Connected to a master server at 127.7.132.190:35599
I20260812 03:47:41.287053 9432 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:41.287194 9432 heartbeater.cc:511] Master 127.7.132.190:35599 requested a full tablet report, sending...
I20260812 03:47:41.287495 8629 ts_manager.cc:194] Registered new tserver with Master: 6466e42574ad4e85b1dadbdf11e72da9 (127.7.132.131:40809)
I20260812 03:47:41.287817 8629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:56367
I20260812 03:47:41.291447 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:47:41.298870 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.293410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.7.132.190:35599)
0812 03:47:41.293450 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.293453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.293572 (+ 119us) server_negotiation.cc:407] Connection header received
0812 03:47:41.293595 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.293596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.293604 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.293654 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:41.294129 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.294553 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.295318 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.295488 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.295540 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.296526 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:41.296527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:41.296528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:41.296533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:41.297910 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.298215 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.298215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.298216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.298263 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.298403 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.298403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.298403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.298517 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:41.298593 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.298665 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.298786 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:47:41.301573 8629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:47:41.303406 8629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:41.309969 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.305921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:38837 (local address 127.0.0.1:55452)
0812 03:47:41.306644 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:41.306652 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:41.306757 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:41.306923 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:41.306924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:41.306926 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:41.307059 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.307061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.308187 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.308188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.308932 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.308934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.309672 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.309673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.309751 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.309764 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:47:41.309847 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:47:41.309899 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":76,"threads_started":1}
I20260812 03:47:41.309978 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.306143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.7.132.129:38837)
0812 03:47:41.306329 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.306333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.306739 (+ 406us) server_negotiation.cc:407] Connection header received
0812 03:47:41.306824 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.306826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.306841 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.306916 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:41.307142 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.308096 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.309030 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.309578 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.309674 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.309732 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.309864 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.309906 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":79,"threads_started":1}
I20260812 03:47:41.310122 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.305899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:40809 (local address 127.0.0.1:56366)
0812 03:47:41.306017 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:41.306053 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:41.306114 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:41.306565 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:41.306566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:41.306569 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:41.306621 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.306622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.307806 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.307807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.308965 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.308968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.309888 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.309889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.309955 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.309959 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:47:41.309990 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 03:47:41.310039 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":5,"thread_start_us":52,"threads_started":1}
I20260812 03:47:41.310313 9459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.306137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.7.132.130:33209)
0812 03:47:41.306278 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.306282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.306292 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:47:41.306348 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.306349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.306360 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.306417 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:41.306593 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.307509 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.309074 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.309784 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.309856 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.309910 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.310173 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.310221 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260812 03:47:41.310436 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.305901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33209 (local address 127.0.0.1:56600)
0812 03:47:41.306219 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:41.306229 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:41.306279 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:41.306456 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:41.306459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:41.306461 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:41.306521 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.306523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.307621 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.307622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.308968 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.308969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.309939 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.309940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.310006 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.310009 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:47:41.310065 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:47:41.310093 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":41,"thread_start_us":70,"threads_started":1}
I20260812 03:47:41.310779 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.306048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.7.132.131:40809)
0812 03:47:41.306394 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.306413 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.306425 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:47:41.306455 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.306457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.306470 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.306548 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:41.306698 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.307715 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.309075 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.309781 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.310559 (+ 778us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.310625 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.310658 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.310694 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":103,"threads_started":1}
I20260812 03:47:41.310916 8845 tablet_service.cc:1467] Processing CreateTablet for tablet c4d1a4b1e7c94538be92176fe35a6289 (DEFAULT_TABLE table=test-table [id=8ec745f55645401a93f17b0fb099cb08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:41.311090 9088 tablet_service.cc:1467] Processing CreateTablet for tablet c4d1a4b1e7c94538be92176fe35a6289 (DEFAULT_TABLE table=test-table [id=8ec745f55645401a93f17b0fb099cb08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:41.311290 8845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d1a4b1e7c94538be92176fe35a6289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:41.311398 9088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d1a4b1e7c94538be92176fe35a6289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:41.314632 9324 tablet_service.cc:1467] Processing CreateTablet for tablet c4d1a4b1e7c94538be92176fe35a6289 (DEFAULT_TABLE table=test-table [id=8ec745f55645401a93f17b0fb099cb08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:41.314821 9462 tablet_bootstrap.cc:492] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4: Bootstrap starting.
I20260812 03:47:41.314991 9324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d1a4b1e7c94538be92176fe35a6289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:41.315442 9462 tablet_bootstrap.cc:654] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:41.316120 9462 log.cc:824] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:41.316787 9464 tablet_bootstrap.cc:492] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea: Bootstrap starting.
I20260812 03:47:41.316934 9462 tablet_bootstrap.cc:492] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4: No bootstrap required, opened a new log
I20260812 03:47:41.317016 9462 ts_tablet_manager.cc:1397] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:47:41.317586 9464 tablet_bootstrap.cc:654] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:41.317965 9465 tablet_bootstrap.cc:492] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9: Bootstrap starting.
I20260812 03:47:41.318154 9464 log.cc:824] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:41.318756 9465 tablet_bootstrap.cc:654] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:41.318868 9464 tablet_bootstrap.cc:492] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea: No bootstrap required, opened a new log
I20260812 03:47:41.318975 9464 ts_tablet_manager.cc:1397] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:47:41.319361 9465 log.cc:824] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:41.319314 9462 raft_consensus.cc:348] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.319499 9462 raft_consensus.cc:374] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:41.319552 9462 raft_consensus.cc:729] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c92337487c44c3788f630c290f94fe4, State: Initialized, Role: FOLLOWER
I20260812 03:47:41.319674 9462 consensus_queue.cc:260] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.319950 9462 ts_tablet_manager.cc:1428] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:47:41.320024 9216 heartbeater.cc:503] Master 127.7.132.190:35599 was elected leader, sending a full tablet report...
I20260812 03:47:41.320794 9464 raft_consensus.cc:348] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.320990 9464 raft_consensus.cc:374] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:41.321043 9464 raft_consensus.cc:729] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae10af3cfb494127bf96ed820154b7ea, State: Initialized, Role: FOLLOWER
I20260812 03:47:41.321159 9464 consensus_queue.cc:260] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.321522 9464 ts_tablet_manager.cc:1428] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:47:41.321553 8960 heartbeater.cc:503] Master 127.7.132.190:35599 was elected leader, sending a full tablet report...
I20260812 03:47:41.322093 9465 tablet_bootstrap.cc:492] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9: No bootstrap required, opened a new log
I20260812 03:47:41.322168 9465 ts_tablet_manager.cc:1397] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260812 03:47:41.323495 9465 raft_consensus.cc:348] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.323627 9465 raft_consensus.cc:374] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:41.323647 9465 raft_consensus.cc:729] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6466e42574ad4e85b1dadbdf11e72da9, State: Initialized, Role: FOLLOWER
I20260812 03:47:41.323711 9465 consensus_queue.cc:260] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.323881 9465 ts_tablet_manager.cc:1428] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:47:41.323927 9432 heartbeater.cc:503] Master 127.7.132.190:35599 was elected leader, sending a full tablet report...
W20260812 03:47:41.362805 9217 tablet.cc:2369] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:47:41.418403 9472 raft_consensus.cc:482] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:41.418538 9472 raft_consensus.cc:504] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.419103 9472 leader_election.cc:288] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c92337487c44c3788f630c290f94fe4 (127.7.132.130:33209), ae10af3cfb494127bf96ed820154b7ea (127.7.132.129:38837)
I20260812 03:47:41.422122 9459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.419378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:55181 (local address 127.7.132.130:33209)
0812 03:47:41.419435 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.419437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.419450 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:47:41.419474 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.419474 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.419483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.419549 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:41.419705 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.420166 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.421296 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.421760 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.421844 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.421853 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.421995 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.422030 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:47:41.422128 9434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.419255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33209 (local address 127.7.132.131:55181)
0812 03:47:41.419302 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:41.419317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:41.419366 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:41.419553 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:41.419554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:41.419556 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:41.419633 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.419635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.420250 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.420251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.421204 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.421207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.421841 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.421842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.421924 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.421928 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:47:41.421989 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:47:41.422030 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":53}
I20260812 03:47:41.422145 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.419388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:54427 (local address 127.7.132.129:38837)
0812 03:47:41.419440 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.419442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.419535 (+ 93us) server_negotiation.cc:407] Connection header received
0812 03:47:41.419614 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.419615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.419627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.419703 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:41.419870 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.420404 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.421388 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.421811 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.421888 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.421896 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.422041 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.422079 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:47:41.422212 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.419276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:38837 (local address 127.7.132.131:54427)
0812 03:47:41.419459 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:41.419469 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:41.419549 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:41.419710 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:41.419711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:41.419712 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:41.419802 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.419804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.420489 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.420490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.421301 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:41.421303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.421889 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.421890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:41.421963 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.421966 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:47:41.422023 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:47:41.422060 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260812 03:47:41.422565 9122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d1a4b1e7c94538be92176fe35a6289" candidate_uuid: "6466e42574ad4e85b1dadbdf11e72da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c92337487c44c3788f630c290f94fe4" is_pre_election: true
I20260812 03:47:41.422844 9122 raft_consensus.cc:2454] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6466e42574ad4e85b1dadbdf11e72da9 in term 0.
I20260812 03:47:41.423126 9252 leader_election.cc:302] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c92337487c44c3788f630c290f94fe4, 6466e42574ad4e85b1dadbdf11e72da9; no voters:
I20260812 03:47:41.423103 8880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d1a4b1e7c94538be92176fe35a6289" candidate_uuid: "6466e42574ad4e85b1dadbdf11e72da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae10af3cfb494127bf96ed820154b7ea" is_pre_election: true
I20260812 03:47:41.423256 8880 raft_consensus.cc:2454] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6466e42574ad4e85b1dadbdf11e72da9 in term 0.
I20260812 03:47:41.423290 9472 raft_consensus.cc:2781] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:41.423364 9472 raft_consensus.cc:482] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:41.423398 9472 raft_consensus.cc:3037] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:41.424072 9472 raft_consensus.cc:504] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.424216 9472 leader_election.cc:288] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [CANDIDATE]: Term 1 election: Requested vote from peers 1c92337487c44c3788f630c290f94fe4 (127.7.132.130:33209), ae10af3cfb494127bf96ed820154b7ea (127.7.132.129:38837)
I20260812 03:47:41.424333 9122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d1a4b1e7c94538be92176fe35a6289" candidate_uuid: "6466e42574ad4e85b1dadbdf11e72da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c92337487c44c3788f630c290f94fe4"
I20260812 03:47:41.424368 8880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d1a4b1e7c94538be92176fe35a6289" candidate_uuid: "6466e42574ad4e85b1dadbdf11e72da9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae10af3cfb494127bf96ed820154b7ea"
I20260812 03:47:41.424430 9122 raft_consensus.cc:3037] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:41.424448 8880 raft_consensus.cc:3037] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:41.425143 8880 raft_consensus.cc:2454] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6466e42574ad4e85b1dadbdf11e72da9 in term 1.
I20260812 03:47:41.425172 9122 raft_consensus.cc:2454] T c4d1a4b1e7c94538be92176fe35a6289 P 1c92337487c44c3788f630c290f94fe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6466e42574ad4e85b1dadbdf11e72da9 in term 1.
I20260812 03:47:41.425323 9253 leader_election.cc:302] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6466e42574ad4e85b1dadbdf11e72da9, ae10af3cfb494127bf96ed820154b7ea; no voters:
I20260812 03:47:41.425419 9472 raft_consensus.cc:2781] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:41.425577 9472 raft_consensus.cc:686] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [term 1 LEADER]: Becoming Leader. State: Replica: 6466e42574ad4e85b1dadbdf11e72da9, State: Running, Role: LEADER
I20260812 03:47:41.425678 9472 consensus_queue.cc:237] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } }
I20260812 03:47:41.426561 8631 catalog_manager.cc:5495] T c4d1a4b1e7c94538be92176fe35a6289 P 6466e42574ad4e85b1dadbdf11e72da9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6466e42574ad4e85b1dadbdf11e72da9 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "6466e42574ad4e85b1dadbdf11e72da9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6466e42574ad4e85b1dadbdf11e72da9" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c92337487c44c3788f630c290f94fe4" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae10af3cfb494127bf96ed820154b7ea" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38837 } health_report { overall_health: UNKNOWN } } }
W20260812 03:47:41.441102 8961 tablet.cc:2369] T c4d1a4b1e7c94538be92176fe35a6289 P ae10af3cfb494127bf96ed820154b7ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:47:41.457324 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.450304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55456 (local address 127.7.132.129:38837)
0812 03:47:41.450378 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.450379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.450448 (+ 69us) server_negotiation.cc:407] Connection header received
0812 03:47:41.450510 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.450511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.450520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.450562 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:41.451207 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.451655 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.452379 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.452557 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.452609 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.454790 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:41.454805 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:41.454805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:41.454811 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:41.456147 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.456517 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.456520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.456521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.456586 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.456785 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.456786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.456786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.456924 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:41.457070 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.457134 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.457218 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506461, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506461, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:47:41.479710 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.473235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55462 (local address 127.7.132.129:38837)
0812 03:47:41.473294 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.473296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.473383 (+ 87us) server_negotiation.cc:407] Connection header received
0812 03:47:41.473461 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.473462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.473476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.473532 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:41.474046 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.474495 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.475240 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.475419 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.475487 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.477453 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:41.477454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:41.477454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:41.477461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:41.478502 (+ 1041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.478924 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.478925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.478925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.478980 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.479177 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.479178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.479178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.479276 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:41.479374 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.479536 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.479613 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[8555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:47:41.501004 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:41.494792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.7.132.129:38837)
0812 03:47:41.494859 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:41.494861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:41.494947 (+ 86us) server_negotiation.cc:407] Connection header received
0812 03:47:41.495023 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:41.495024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:41.495032 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:41.495096 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:41.495594 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.496027 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.496695 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:41.496865 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:41.496931 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:41.498809 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:41.498810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:41.498811 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:41.498817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:41.499934 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.500281 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.500282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.500283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.500333 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:41.500542 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:41.500543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:41.500544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:41.500637 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:41.500731 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:41.500838 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 03:47:41.500907 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 03:47:41.501645 8811 tablet_service.cc:2908] Rejecting scan request for tablet c4d1a4b1e7c94538be92176fe35a6289: Uninitialized: clean time has not yet been initialized
I20260812 03:47:41.502775 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8741
I20260812 03:47:41.515673 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8970
I20260812 03:47:41.523779 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9225
I20260812 03:47:41.531560 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8577
2026-08-12T03:47:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2204 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[9495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[9495](info): set up 2 sockets
Aug 12 03:47:41 dist-test-slave-39l8 krb5kdc[9495](info): commencing operation
krb5kdc: starting...
W20260812 03:47:42.735805 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:47:42 dist-test-slave-39l8 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:47:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:47:42.847151 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37233
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:37233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:42.932417 9522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:42.932614 9522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:42.932636 9522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:42.933665 9522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:47:42.933698 9522 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:47:42.933710 9522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:42.933723 9522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:42.933735 9522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:47:42.933746 9522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:47:42.935148 9522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:37233
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:42.935362 9522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:42.935539 9522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:42.940244 9530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:42.940241 9532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:42.940258 9534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:42.940431 9522 server_base.cc:1034] running on GCE node
I20260812 03:47:42.940665 9522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:42.940953 9522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:42.942094 9522 hybrid_clock.cc:648] HybridClock initialized: now 1786506462942073 us; error 37 us; skew 500 ppm
Aug 12 03:47:42 dist-test-slave-39l8 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506462, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:42.944460 9522 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:47:42.944886 9522 webserver.cc:460] Webserver started at http://127.7.132.190:32867/ using document root <none> and password file <none>
I20260812 03:47:42.945109 9522 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:42.945173 9522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:42.945304 9522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:42.946328 9522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "cdb73a044353437191336fd6781c9867"
format_stamp: "Formatted at 2026-08-12 03:47:42 on dist-test-slave-39l8"
server_key: "0300abecea1d51caa003f3e181faaa1d"
server_key_iv: "3ab127a21c401d9cfa450c24bfe81222"
server_key_version: "encryptionkey@0"
I20260812 03:47:42.946748 9522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "cdb73a044353437191336fd6781c9867"
format_stamp: "Formatted at 2026-08-12 03:47:42 on dist-test-slave-39l8"
server_key: "0300abecea1d51caa003f3e181faaa1d"
server_key_iv: "3ab127a21c401d9cfa450c24bfe81222"
server_key_version: "encryptionkey@0"
I20260812 03:47:42.948101 9522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:42.948810 9543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:42.948984 9522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:42.949105 9522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "cdb73a044353437191336fd6781c9867"
format_stamp: "Formatted at 2026-08-12 03:47:42 on dist-test-slave-39l8"
server_key: "0300abecea1d51caa003f3e181faaa1d"
server_key_iv: "3ab127a21c401d9cfa450c24bfe81222"
server_key_version: "encryptionkey@0"
I20260812 03:47:42.949193 9522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:42.957047 9522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:42.958841 9522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:42.958982 9522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:42.964088 9522 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:37233
I20260812 03:47:42.964131 9648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:37233 every 8 connection(s)
I20260812 03:47:42.964478 9522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:47:42.965121 9649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:42.967723 9649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867: Bootstrap starting.
I20260812 03:47:42.968387 9649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:42.968691 9649 log.cc:824] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:42.969386 9649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867: No bootstrap required, opened a new log
I20260812 03:47:42.971966 9649 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb73a044353437191336fd6781c9867" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37233 } }
I20260812 03:47:42.972229 9649 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:42.972259 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9522
I20260812 03:47:42.972273 9649 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb73a044353437191336fd6781c9867, State: Initialized, Role: FOLLOWER
I20260812 03:47:42.972357 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:47:42.972433 9649 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb73a044353437191336fd6781c9867" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37233 } }
I20260812 03:47:42.972520 7698 external_mini_cluster.cc:1428] Setting key 292a81c6c0377be08a29d9cbabd08037
I20260812 03:47:42.972522 9649 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:47:42.972556 9649 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:47:42.972741 9649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:42.973501 9649 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb73a044353437191336fd6781c9867" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37233 } }
I20260812 03:47:42.973634 9649 leader_election.cc:302] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdb73a044353437191336fd6781c9867; no voters:
I20260812 03:47:42.973819 9649 leader_election.cc:288] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:47:42.973869 9659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:42.974025 9659 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [term 1 LEADER]: Becoming Leader. State: Replica: cdb73a044353437191336fd6781c9867, State: Running, Role: LEADER
I20260812 03:47:42.974092 9649 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:47:42.974112 9659 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb73a044353437191336fd6781c9867" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37233 } }
I20260812 03:47:42.974448 9659 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdb73a044353437191336fd6781c9867. Latest consensus state: current_term: 1 leader_uuid: "cdb73a044353437191336fd6781c9867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb73a044353437191336fd6781c9867" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37233 } } }
I20260812 03:47:42.974534 9659 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:42.974464 9666 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdb73a044353437191336fd6781c9867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdb73a044353437191336fd6781c9867" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37233 } } }
I20260812 03:47:42.974745 9666 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:42.974872 9677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:47:42.975536 9677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:47:42.977761 9677 catalog_manager.cc:1347] Generated new cluster ID: 37315729133f42d6abc1d5e3994a7b6b
I20260812 03:47:42.977831 9677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:47:42 dist-test-slave-39l8 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:42.985088 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:42.973164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.7.132.190:37233)
0812 03:47:42.973364 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:42.973374 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:42.973422 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:47:42.973515 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:42.973520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:42.973561 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:42.973701 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:42.974572 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:42.976058 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:42.977135 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:42.977716 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:42.977839 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:42.980811 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:42.980815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:42.980815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:42.980828 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:42.983464 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:42.983876 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:42.983879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:42.983879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:42.983978 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:42.984195 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:42.984195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:42.984196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:42.984495 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:42.984644 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:42.984749 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 03:47:42.984887 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260812 03:47:42.995328 9677 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:47:42.996057 9677 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:47:43.000253 9677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867: Generated new TSK 0
I20260812 03:47:43.000430 9677 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:47:43.029441 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37233
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:43.109329 9697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:43.109544 9697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:43.109584 9697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:43.110700 9697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:43.110795 9697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:43.110826 9697 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:47:43.110857 9697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:47:43.112264 9697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:37233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:43.112524 9697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:43.112747 9697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:43.117583 9709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:43.117619 9704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:43.117651 9706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:43.117755 9697 server_base.cc:1034] running on GCE node
I20260812 03:47:43.117935 9697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:43.118156 9697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:43.119306 9697 hybrid_clock.cc:648] HybridClock initialized: now 1786506463119295 us; error 41 us; skew 500 ppm
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:43.122318 9697 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:47:43.122859 9697 webserver.cc:460] Webserver started at http://127.7.132.129:39471/ using document root <none> and password file <none>
I20260812 03:47:43.123167 9697 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:43.123235 9697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:43.123365 9697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:43.124475 9697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "68e73537ce65472c89ead196cd254fba"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "f3306c670a3727569f5acb7d76ba3025"
server_key_iv: "3d121bf2d944fbd1d1f38d1fd63d9431"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.124837 9697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "68e73537ce65472c89ead196cd254fba"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "f3306c670a3727569f5acb7d76ba3025"
server_key_iv: "3d121bf2d944fbd1d1f38d1fd63d9431"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.126163 9697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:47:43.126884 9722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.127056 9697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:43.127156 9697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "68e73537ce65472c89ead196cd254fba"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "f3306c670a3727569f5acb7d76ba3025"
server_key_iv: "3d121bf2d944fbd1d1f38d1fd63d9431"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.127238 9697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:43.141242 9697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:43.142906 9697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:43.143067 9697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:43.143555 9697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:43.143611 9697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.143643 9697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:43.143673 9697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.150446 9697 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:37759
I20260812 03:47:43.150511 9904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:37759 every 8 connection(s)
I20260812 03:47:43.150833 9697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:43.154016 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.144193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:48977 (local address 127.7.132.190:37233)
0812 03:47:43.144245 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.144247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.144468 (+ 221us) server_negotiation.cc:407] Connection header received
0812 03:47:43.144944 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.144945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.144960 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.145031 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:43.145985 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.146852 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.148372 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.148634 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.148717 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.151414 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:43.151416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:43.151416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:43.151424 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:43.152678 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.153103 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.153104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.153105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.153165 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.153431 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.153432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.153432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.153537 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:43.153605 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.153851 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.153935 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:47:43.154147 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.144190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37233 (local address 127.7.132.129:48977)
0812 03:47:43.144363 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.144395 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.144848 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.145150 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.145155 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.145564 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:43.145864 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.145871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.146952 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.146953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.148300 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.148303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.148729 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.148730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.148877 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.149449 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:43.149461 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:43.151316 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:43.152781 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.152806 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.152808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.153015 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.153247 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.153248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.153249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.153351 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.153595 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:43.153601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:43.153723 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.153817 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.153953 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260812 03:47:43.154359 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9697
I20260812 03:47:43.154448 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:47:43.154626 7698 external_mini_cluster.cc:1428] Setting key d91a464d201d0d7cb570e1575c901a0f
I20260812 03:47:43.154628 9908 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37233
I20260812 03:47:43.154724 9908 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:43.154927 9908 heartbeater.cc:511] Master 127.7.132.190:37233 requested a full tablet report, sending...
I20260812 03:47:43.155542 9572 ts_manager.cc:194] Registered new tserver with Master: 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759)
I20260812 03:47:43.156345 9572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:48977
I20260812 03:47:43.162323 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.156539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:55807 (local address 127.7.132.190:37233)
0812 03:47:43.156617 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.156619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.156626 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:47:43.156643 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.156644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.156652 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.156691 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:43.157076 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.157538 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.158415 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.158679 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.158768 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.159801 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:43.159803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:43.159803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:43.159811 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:43.161147 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.161535 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.161536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.161536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.161616 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.161900 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.161901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.161901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.161995 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:43.162049 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.162206 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.162270 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:47:43.162391 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.155715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37233 (local address 127.7.132.129:55807)
0812 03:47:43.155848 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.155858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.155902 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.156734 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.156735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.156929 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:43.156995 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.156996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.157632 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.157633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.158334 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.158337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.158781 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.158782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.158859 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.159273 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:43.159278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:43.159694 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:43.161228 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.161229 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.161229 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.161461 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.161764 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.161765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.161765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.161834 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.162063 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:43.162063 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:43.162116 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.162190 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.162287 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:47:43.180775 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37233
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:43.261013 9926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:43.261248 9926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:43.261289 9926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:43.262380 9926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:43.262447 9926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:43.262478 9926 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:47:43.262507 9926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:47:43.263888 9926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:37233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:43.264134 9926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:43.264348 9926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:47:43.269181 9926 server_base.cc:1034] running on GCE node
W20260812 03:47:43.269163 9943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:43.269161 9939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:43.269155 9938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:43.269574 9926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:43.269841 9926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:43.270985 9926 hybrid_clock.cc:648] HybridClock initialized: now 1786506463270971 us; error 33 us; skew 500 ppm
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:43.273355 9926 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:47:43.273798 9926 webserver.cc:460] Webserver started at http://127.7.132.130:33001/ using document root <none> and password file <none>
I20260812 03:47:43.273968 9926 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:43.274006 9926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:43.274125 9926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:43.275209 9926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "834e849fd0174d968aca09501610477d"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "3e34f9b08cb3d015988630059284bcb8"
server_key_iv: "c518eae4f6b85753d81962fde7c5cd0f"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.275604 9926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "834e849fd0174d968aca09501610477d"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "3e34f9b08cb3d015988630059284bcb8"
server_key_iv: "c518eae4f6b85753d81962fde7c5cd0f"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.276871 9926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:47:43.277676 9955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.277850 9926 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:43.277928 9926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "834e849fd0174d968aca09501610477d"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "3e34f9b08cb3d015988630059284bcb8"
server_key_iv: "c518eae4f6b85753d81962fde7c5cd0f"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.277979 9926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:43.294144 9926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:43.296355 9926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:43.296459 9926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:43.296887 9926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:43.296924 9926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.296944 9926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:43.296996 9926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.303818 9926 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37949
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:43.303889 10130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37949 every 8 connection(s)
I20260812 03:47:43.304214 9926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:47:43.306185 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9926
I20260812 03:47:43.306308 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:47:43.306479 7698 external_mini_cluster.cc:1428] Setting key 141ed39aa699fa3fb2ac1a2fb8ae9692
I20260812 03:47:43.306854 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.297359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:49357 (local address 127.7.132.190:37233)
0812 03:47:43.297427 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.297430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.297576 (+ 146us) server_negotiation.cc:407] Connection header received
0812 03:47:43.298019 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.298020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.298033 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.298118 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:43.298918 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.299775 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.301342 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.301693 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.301793 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.304419 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:43.304420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:43.304421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:43.304429 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:43.305555 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.305885 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.305886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.305886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.305931 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.306129 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.306131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.306131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.306251 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:43.306347 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.306652 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.306777 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:47:43.307011 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.297313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37233 (local address 127.7.132.130:49357)
0812 03:47:43.297457 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.297495 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.297930 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.298164 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.298167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.298505 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:43.298817 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.298824 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.299878 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.299880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.301206 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.301209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.301812 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.301813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.301939 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.302448 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:43.302459 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:43.304313 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:43.305618 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.305623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.305625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.305826 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.305969 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.305970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.305970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.306045 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.306343 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:43.306350 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:43.306502 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.306619 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.306813 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260812 03:47:43.307529 10131 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37233
I20260812 03:47:43.307610 10131 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:43.307782 10131 heartbeater.cc:511] Master 127.7.132.190:37233 requested a full tablet report, sending...
I20260812 03:47:43.308127 9571 ts_manager.cc:194] Registered new tserver with Master: 834e849fd0174d968aca09501610477d (127.7.132.130:37949)
I20260812 03:47:43.308622 9571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:49357
I20260812 03:47:43.314517 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.308556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:50759 (local address 127.7.132.190:37233)
0812 03:47:43.308594 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.308596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.308716 (+ 120us) server_negotiation.cc:407] Connection header received
0812 03:47:43.308794 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.308795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.308805 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.308856 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:43.309308 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.309805 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.310858 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.311022 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.311072 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.312015 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:43.312016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:43.312017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:43.312030 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:43.313360 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.313751 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.313752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.313752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.313799 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.313972 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.313973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.313973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.314107 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:43.314207 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.314359 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.314431 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:47:43.314538 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.308513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37233 (local address 127.7.132.130:50759)
0812 03:47:43.308643 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.308652 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.308743 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.308899 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.308900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.309154 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:43.309219 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.309221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.309911 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.309912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.310784 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.310787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.311088 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.311089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.311166 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.311520 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:43.311526 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:43.311926 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:43.313437 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.313438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.313439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.313669 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.313851 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.313852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.313852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.313917 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.314203 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:43.314204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:43.314251 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.314334 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.314433 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:47:43.331113 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37233
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:43.411202 10148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:43.411419 10148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:43.411455 10148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:43.412608 10148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:43.412669 10148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:43.412703 10148 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:47:43.412741 10148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:47:43.414157 10148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:37233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:43.414412 10148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:43.414633 10148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:43.419512 10160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:43.419528 10158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:43.419553 10157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:43.419770 10148 server_base.cc:1034] running on GCE node
I20260812 03:47:43.419962 10148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:43.420150 10148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:43.421283 10148 hybrid_clock.cc:648] HybridClock initialized: now 1786506463421273 us; error 27 us; skew 500 ppm
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:43.423770 10148 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:47:43.424172 10148 webserver.cc:460] Webserver started at http://127.7.132.131:32843/ using document root <none> and password file <none>
I20260812 03:47:43.424399 10148 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:43.424443 10148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:43.424597 10148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:43.425609 10148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "6c914d81bf295b514ae9b742d477da1c"
server_key_iv: "1e5e14cde069ba0691e8b728b2415028"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.425961 10148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "6c914d81bf295b514ae9b742d477da1c"
server_key_iv: "1e5e14cde069ba0691e8b728b2415028"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.427368 10148 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:47:43.428073 10170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.428233 10148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:43.428332 10148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed"
format_stamp: "Formatted at 2026-08-12 03:47:43 on dist-test-slave-39l8"
server_key: "6c914d81bf295b514ae9b742d477da1c"
server_key_iv: "1e5e14cde069ba0691e8b728b2415028"
server_key_version: "encryptionkey@0"
I20260812 03:47:43.428401 10148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:43.444869 10148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:43.446870 10148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:43.447052 10148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:43.447564 10148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:43.447605 10148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.447628 10148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:43.447682 10148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:43.454762 10148 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:34607
I20260812 03:47:43.454826 10367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:34607 every 8 connection(s)
I20260812 03:47:43.455179 10148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:47:43.456022 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10148
I20260812 03:47:43.456112 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:47:43.456290 7698 external_mini_cluster.cc:1428] Setting key 46bb67ab9503717b60c39d68fe5df036
I20260812 03:47:43.457834 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.448061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:50541 (local address 127.7.132.190:37233)
0812 03:47:43.448121 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.448124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.448446 (+ 322us) server_negotiation.cc:407] Connection header received
0812 03:47:43.449051 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.449052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.449067 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.449184 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:43.450022 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.450884 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.452004 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.452432 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.452514 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.455066 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:43.455068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:43.455068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:43.455075 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:43.456743 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.457189 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.457190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.457190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.457239 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.457414 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.457414 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.457414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.457501 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:43.457544 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.457721 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.457776 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:47:43.457996 10180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.448019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37233 (local address 127.7.132.131:50541)
0812 03:47:43.448307 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.448348 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.448930 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.449264 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.449268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.449634 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:43.449912 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.449918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.450987 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.450988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.451915 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.451919 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.452540 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.452541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.452677 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.453203 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:43.453211 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:43.454968 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:43.456812 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.456820 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.456822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.457116 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.457285 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.457285 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.457285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.457375 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.457534 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:43.457536 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:43.457633 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.457709 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.457816 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":151,"threads_started":1}
I20260812 03:47:43.458449 10368 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37233
I20260812 03:47:43.458532 10368 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:43.458755 10368 heartbeater.cc:511] Master 127.7.132.190:37233 requested a full tablet report, sending...
I20260812 03:47:43.459163 9571 ts_manager.cc:194] Registered new tserver with Master: 4737b54ef8564fcc8fe3ba0a5753b7ed (127.7.132.131:34607)
I20260812 03:47:43.459570 9571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:50541
I20260812 03:47:43.464802 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.459518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37233 (local address 127.7.132.131:54695)
0812 03:47:43.459770 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.459780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.459853 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.460393 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.460394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.460577 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:43.460622 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.460624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.461197 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.461198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.461673 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.461674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.462001 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.462002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.462068 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.462402 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:43.462406 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:43.462657 (+ 251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:43.463868 (+ 1211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.463868 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.463869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.464026 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.464228 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.464228 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.464229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.464286 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.464506 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:43.464507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:43.464547 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.464628 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.464717 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":77,"threads_started":1}
I20260812 03:47:43.464797 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.459596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:54695 (local address 127.7.132.190:37233)
0812 03:47:43.459639 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.459641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.459841 (+ 200us) server_negotiation.cc:407] Connection header received
0812 03:47:43.460306 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.460307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.460315 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.460388 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:43.460690 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.461105 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.461757 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.461940 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.462002 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.462757 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:43.462758 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:43.462758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:43.462765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:43.463795 (+ 1030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.464106 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.464107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.464107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.464161 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.464346 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.464346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.464347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.464442 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:43.464511 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.464652 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.464715 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:47:43.467761 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:43.491140 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.484433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37924 (local address 127.7.132.190:37233)
0812 03:47:43.484490 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.484492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.484501 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:47:43.484550 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.484551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.484560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.484608 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:43.485165 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.485633 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.486310 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.486489 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.486560 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.488619 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:43.488621 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:43.488621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:43.488628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:43.489760 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.490085 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.490086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.490087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.490177 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.490448 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.490449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.490450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.490605 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:43.490735 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.490921 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.491018 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:47:43.492771 9571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:47:43.494364 9571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:43.500464 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.496412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34607 (local address 127.0.0.1:38536)
0812 03:47:43.496683 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.496692 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.496738 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.497050 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.497051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.497053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:43.497107 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.497109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.498350 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.498351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.499466 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.499468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.500124 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.500128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.500207 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.500220 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.500328 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.500381 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":52,"threads_started":1}
I20260812 03:47:43.500474 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.496696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38536 (local address 127.7.132.131:34607)
0812 03:47:43.496876 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.496881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.496889 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:47:43.496929 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.496930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.496941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.497052 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:43.497210 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.498267 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.499555 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.500036 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.500123 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.500189 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.500346 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.500383 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260812 03:47:43.500932 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.496602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:37759 (local address 127.0.0.1:37046)
0812 03:47:43.496882 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.496892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.496936 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.497391 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.497391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.497393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:43.497429 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.497431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.498572 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.498573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.499552 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.499555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.500492 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.500493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.500728 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.500734 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.500791 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.500851 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":38,"threads_started":1}
I20260812 03:47:43.501013 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.496515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.7.132.130:37949)
0812 03:47:43.496827 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.496831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.496841 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:47:43.496879 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.496881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.496897 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.496996 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:43.497183 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.498000 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.499510 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.500264 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.500350 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.500408 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.500871 (+ 463us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.500914 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":74,"threads_started":1}
I20260812 03:47:43.500954 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.496408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37949 (local address 127.0.0.1:33420)
0812 03:47:43.496566 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.496605 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.496681 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.497013 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.497015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.497018 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:43.497083 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.497085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.498076 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.498077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.499419 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.499422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.500447 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.500448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.500782 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.500786 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.500850 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.500879 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":29,"thread_start_us":62,"threads_started":1}
I20260812 03:47:43.501291 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.496706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37046 (local address 127.7.132.129:37759)
0812 03:47:43.497174 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.497178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.497188 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:47:43.497224 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.497227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.497255 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.497325 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:43.497517 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.498492 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.499648 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.500358 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.500416 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.500464 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.501132 (+ 668us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.501181 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":96,"threads_started":1}
I20260812 03:47:43.501192 10261 tablet_service.cc:1467] Processing CreateTablet for tablet a4b030de2bf64295abf05d16fb136ed6 (DEFAULT_TABLE table=test-table [id=4d6bfb7764b749f99d2f177f72d362b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:43.501565 10261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b030de2bf64295abf05d16fb136ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.501852 9806 tablet_service.cc:1467] Processing CreateTablet for tablet a4b030de2bf64295abf05d16fb136ed6 (DEFAULT_TABLE table=test-table [id=4d6bfb7764b749f99d2f177f72d362b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:43.502218 9806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b030de2bf64295abf05d16fb136ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.502149 10025 tablet_service.cc:1467] Processing CreateTablet for tablet a4b030de2bf64295abf05d16fb136ed6 (DEFAULT_TABLE table=test-table [id=4d6bfb7764b749f99d2f177f72d362b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:43.502492 10025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b030de2bf64295abf05d16fb136ed6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.504498 10399 tablet_bootstrap.cc:492] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Bootstrap starting.
I20260812 03:47:43.505306 10399 tablet_bootstrap.cc:654] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.505597 10399 log.cc:824] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:43.505852 10400 tablet_bootstrap.cc:492] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Bootstrap starting.
I20260812 03:47:43.506479 10400 tablet_bootstrap.cc:654] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.506821 10400 log.cc:824] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:43.506882 10402 tablet_bootstrap.cc:492] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Bootstrap starting.
I20260812 03:47:43.507337 10402 tablet_bootstrap.cc:654] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.507464 10400 tablet_bootstrap.cc:492] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: No bootstrap required, opened a new log
I20260812 03:47:43.507560 10400 ts_tablet_manager.cc:1397] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:47:43.507666 10402 log.cc:824] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:43.508338 10399 tablet_bootstrap.cc:492] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: No bootstrap required, opened a new log
I20260812 03:47:43.508338 10402 tablet_bootstrap.cc:492] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: No bootstrap required, opened a new log
I20260812 03:47:43.508404 10399 ts_tablet_manager.cc:1397] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:47:43.508404 10402 ts_tablet_manager.cc:1397] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:47:43.509191 10400 raft_consensus.cc:348] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.509363 10400 raft_consensus.cc:374] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.509410 10400 raft_consensus.cc:729] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e73537ce65472c89ead196cd254fba, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.509577 10400 consensus_queue.cc:260] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.509856 10400 ts_tablet_manager.cc:1428] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 03:47:43.509999 9908 heartbeater.cc:503] Master 127.7.132.190:37233 was elected leader, sending a full tablet report...
I20260812 03:47:43.510424 10399 raft_consensus.cc:348] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.510605 10399 raft_consensus.cc:374] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.510641 10399 raft_consensus.cc:729] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4737b54ef8564fcc8fe3ba0a5753b7ed, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.510572 10402 raft_consensus.cc:348] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.510754 10402 raft_consensus.cc:374] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.510800 10402 raft_consensus.cc:729] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834e849fd0174d968aca09501610477d, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.510774 10399 consensus_queue.cc:260] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.510947 10402 consensus_queue.cc:260] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.511034 10399 ts_tablet_manager.cc:1428] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:47:43.511165 10368 heartbeater.cc:503] Master 127.7.132.190:37233 was elected leader, sending a full tablet report...
I20260812 03:47:43.511222 10402 ts_tablet_manager.cc:1428] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:47:43.511267 10131 heartbeater.cc:503] Master 127.7.132.190:37233 was elected leader, sending a full tablet report...
W20260812 03:47:43.555022 10132 tablet.cc:2369] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:47:43.651721 9909 tablet.cc:2369] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:47:43.705919 10369 tablet.cc:2369] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:47:43.823202 10407 raft_consensus.cc:482] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:43.823341 10407 raft_consensus.cc:504] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.823784 10407 leader_election.cc:288] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 834e849fd0174d968aca09501610477d (127.7.132.130:37949), 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759)
I20260812 03:47:43.826495 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.824067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:58895 (local address 127.7.132.129:37759)
0812 03:47:43.824108 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.824110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.824131 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:47:43.824159 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.824160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.824172 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.824258 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:43.824495 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.824976 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.825802 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.826116 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.826208 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.826217 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.826367 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.826422 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:47:43.826514 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.824022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49247 (local address 127.7.132.130:37949)
0812 03:47:43.824079 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.824081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.824220 (+ 139us) server_negotiation.cc:407] Connection header received
0812 03:47:43.824268 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.824269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.824278 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.824330 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:43.824483 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.824938 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.825806 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.826125 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.826215 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.826223 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.826407 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.826444 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:47:43.826547 10180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.823976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:37759 (local address 127.7.132.131:58895)
0812 03:47:43.824033 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.824045 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.824087 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.824371 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.824371 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.824373 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:43.824420 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.824422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.825049 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.825049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.825707 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.825709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.826206 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.826207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.826287 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.826290 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.826360 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.826422 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:47:43.826781 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.823986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37949 (local address 127.7.132.131:49247)
0812 03:47:43.824147 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.824157 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.824212 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.824350 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.824351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.824352 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:43.824414 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.824416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.825014 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.825015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.825711 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.825713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.826216 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.826217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.826299 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.826302 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.826380 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.826441 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260812 03:47:43.826936 9837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b030de2bf64295abf05d16fb136ed6" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e73537ce65472c89ead196cd254fba" is_pre_election: true
I20260812 03:47:43.827077 9837 raft_consensus.cc:2454] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 0.
I20260812 03:47:43.827090 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b030de2bf64295abf05d16fb136ed6" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "834e849fd0174d968aca09501610477d" is_pre_election: true
I20260812 03:47:43.827279 10060 raft_consensus.cc:2454] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 0.
I20260812 03:47:43.827353 10173 leader_election.cc:302] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4737b54ef8564fcc8fe3ba0a5753b7ed, 68e73537ce65472c89ead196cd254fba; no voters:
I20260812 03:47:43.827481 10407 raft_consensus.cc:2781] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:43.827531 10407 raft_consensus.cc:482] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:43.827564 10407 raft_consensus.cc:3037] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.828465 10407 raft_consensus.cc:504] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.828603 10407 leader_election.cc:288] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 election: Requested vote from peers 834e849fd0174d968aca09501610477d (127.7.132.130:37949), 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759)
I20260812 03:47:43.828768 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b030de2bf64295abf05d16fb136ed6" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "834e849fd0174d968aca09501610477d"
I20260812 03:47:43.828768 9837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b030de2bf64295abf05d16fb136ed6" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e73537ce65472c89ead196cd254fba"
I20260812 03:47:43.828851 10060 raft_consensus.cc:3037] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.828848 9837 raft_consensus.cc:3037] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.829563 10060 raft_consensus.cc:2454] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 1.
I20260812 03:47:43.829669 9837 raft_consensus.cc:2454] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 1.
I20260812 03:47:43.829746 10172 leader_election.cc:302] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4737b54ef8564fcc8fe3ba0a5753b7ed, 834e849fd0174d968aca09501610477d; no voters:
I20260812 03:47:43.829891 10407 raft_consensus.cc:2781] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:43.830037 10407 raft_consensus.cc:686] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 LEADER]: Becoming Leader. State: Replica: 4737b54ef8564fcc8fe3ba0a5753b7ed, State: Running, Role: LEADER
I20260812 03:47:43.830142 10407 consensus_queue.cc:237] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } }
I20260812 03:47:43.831090 9572 catalog_manager.cc:5495] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 4737b54ef8564fcc8fe3ba0a5753b7ed (127.7.132.131). New cstate: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } health_report { overall_health: UNKNOWN } } }
I20260812 03:47:43.855178 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.852757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.7.132.131:34607)
0812 03:47:43.852818 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.852821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.852862 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:47:43.853000 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.853001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.853018 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.853076 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:47:43.853196 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.853634 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.854313 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.854475 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.854546 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.854819 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:47:43.854912 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:47:43.854990 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.855054 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.855075 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:47:43.856998 10060 raft_consensus.cc:1261] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Refusing update from remote peer 4737b54ef8564fcc8fe3ba0a5753b7ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:43.857352 10411 consensus_queue.cc:1035] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:43.858423 9837 raft_consensus.cc:1261] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Refusing update from remote peer 4737b54ef8564fcc8fe3ba0a5753b7ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:43.858690 10411 consensus_queue.cc:1035] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:43.859750 10419 mvcc.cc:205] Tried to move back new op lower bound from 7317530475955175424 to 7317530475848986624. Current Snapshot: MvccSnapshot[applied={T|T < 7317530475955175424}]
I20260812 03:47:43.860098 10421 mvcc.cc:205] Tried to move back new op lower bound from 7317530475955175424 to 7317530475848986624. Current Snapshot: MvccSnapshot[applied={T|T < 7317530475955175424}]
I20260812 03:47:43.864723 9572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:43.864877 9572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:43.865736 9572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867: Sending DeleteTablet for 3 replicas of tablet a4b030de2bf64295abf05d16fb136ed6
I20260812 03:47:43.866160 10025 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b030de2bf64295abf05d16fb136ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:43 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:33420
I20260812 03:47:43.866160 9806 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b030de2bf64295abf05d16fb136ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:43 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:37046
I20260812 03:47:43.866397 10432 tablet_replica.cc:330] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: stopping tablet replica
I20260812 03:47:43.866442 10261 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b030de2bf64295abf05d16fb136ed6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:43 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:38536
I20260812 03:47:43.866498 10432 raft_consensus.cc:2229] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:43.866585 10432 raft_consensus.cc:2258] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:43.866604 9572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:47:43.866751 10433 tablet_replica.cc:330] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: stopping tablet replica
W20260812 03:47:43.866808 9572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:43.866873 10433 raft_consensus.cc:2229] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:43.866976 10433 raft_consensus.cc:2258] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:43.867455 10434 tablet_replica.cc:330] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: stopping tablet replica
I20260812 03:47:43.867552 10434 raft_consensus.cc:2229] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:47:43.867645 10434 raft_consensus.cc:2258] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:43.867640 10432 ts_tablet_manager.cc:1905] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:43.868095 10434 ts_tablet_manager.cc:1905] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:43.868144 10025 tablet_service.cc:1467] Processing CreateTablet for tablet 775ecd746bca4dc785bad2e5c879a3f5 (DEFAULT_TABLE table=test-table [id=17cedfffe18246b6948a696470a2afa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:43.868168 9806 tablet_service.cc:1467] Processing CreateTablet for tablet 775ecd746bca4dc785bad2e5c879a3f5 (DEFAULT_TABLE table=test-table [id=17cedfffe18246b6948a696470a2afa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:43.868254 10025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 775ecd746bca4dc785bad2e5c879a3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.868291 9806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 775ecd746bca4dc785bad2e5c879a3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.869302 10432 ts_tablet_manager.cc:1918] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:47:43.869367 10432 log.cc:1196] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/a4b030de2bf64295abf05d16fb136ed6
I20260812 03:47:43.869486 10402 tablet_bootstrap.cc:492] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: Bootstrap starting.
I20260812 03:47:43.869638 10432 ts_tablet_manager.cc:1939] T a4b030de2bf64295abf05d16fb136ed6 P 834e849fd0174d968aca09501610477d: Deleting consensus metadata
I20260812 03:47:43.869719 10400 tablet_bootstrap.cc:492] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: Bootstrap starting.
I20260812 03:47:43.869948 10402 tablet_bootstrap.cc:654] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.870059 9549 catalog_manager.cc:4841] TS 834e849fd0174d968aca09501610477d (127.7.132.130:37949): tablet a4b030de2bf64295abf05d16fb136ed6 (table test-table [id=4d6bfb7764b749f99d2f177f72d362b9]) successfully deleted
I20260812 03:47:43.870232 10400 tablet_bootstrap.cc:654] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.870245 10433 ts_tablet_manager.cc:1905] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:43.870623 10434 ts_tablet_manager.cc:1918] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:47:43.870734 10434 log.cc:1196] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/a4b030de2bf64295abf05d16fb136ed6
I20260812 03:47:43.870806 10261 tablet_service.cc:1467] Processing CreateTablet for tablet 775ecd746bca4dc785bad2e5c879a3f5 (DEFAULT_TABLE table=test-table [id=17cedfffe18246b6948a696470a2afa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:43.870900 10261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 775ecd746bca4dc785bad2e5c879a3f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.871006 10434 ts_tablet_manager.cc:1939] T a4b030de2bf64295abf05d16fb136ed6 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Deleting consensus metadata
I20260812 03:47:43.871771 10433 ts_tablet_manager.cc:1918] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:47:43.871835 10433 log.cc:1196] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/a4b030de2bf64295abf05d16fb136ed6
I20260812 03:47:43.871942 10399 tablet_bootstrap.cc:492] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Bootstrap starting.
I20260812 03:47:43.872006 9554 catalog_manager.cc:4841] TS 4737b54ef8564fcc8fe3ba0a5753b7ed (127.7.132.131:34607): tablet a4b030de2bf64295abf05d16fb136ed6 (table test-table [id=4d6bfb7764b749f99d2f177f72d362b9]) successfully deleted
I20260812 03:47:43.872117 10433 ts_tablet_manager.cc:1939] T a4b030de2bf64295abf05d16fb136ed6 P 68e73537ce65472c89ead196cd254fba: Deleting consensus metadata
I20260812 03:47:43.872473 10399 tablet_bootstrap.cc:654] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.872856 9554 catalog_manager.cc:4841] TS 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759): tablet a4b030de2bf64295abf05d16fb136ed6 (table test-table [id=4d6bfb7764b749f99d2f177f72d362b9]) successfully deleted
I20260812 03:47:43.873100 10399 tablet_bootstrap.cc:492] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: No bootstrap required, opened a new log
I20260812 03:47:43.873107 10400 tablet_bootstrap.cc:492] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: No bootstrap required, opened a new log
I20260812 03:47:43.873157 10399 ts_tablet_manager.cc:1397] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:43.873167 10400 ts_tablet_manager.cc:1397] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 03:47:43.873319 10399 raft_consensus.cc:348] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.873319 10400 raft_consensus.cc:348] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.873389 10399 raft_consensus.cc:374] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.873389 10400 raft_consensus.cc:374] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.873411 10399 raft_consensus.cc:729] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4737b54ef8564fcc8fe3ba0a5753b7ed, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.873412 10400 raft_consensus.cc:729] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e73537ce65472c89ead196cd254fba, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.873456 10400 consensus_queue.cc:260] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.873468 10399 consensus_queue.cc:260] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.873551 10399 ts_tablet_manager.cc:1428] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:43.873562 10400 ts_tablet_manager.cc:1428] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.874035 10402 tablet_bootstrap.cc:492] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: No bootstrap required, opened a new log
I20260812 03:47:43.874114 10402 ts_tablet_manager.cc:1397] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 03:47:43.874325 10402 raft_consensus.cc:348] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.874415 10402 raft_consensus.cc:374] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.874430 10402 raft_consensus.cc:729] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834e849fd0174d968aca09501610477d, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.874496 10402 consensus_queue.cc:260] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.874862 10402 ts_tablet_manager.cc:1428] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:43.894794 10407 raft_consensus.cc:482] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:43.894868 10407 raft_consensus.cc:504] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.895052 10407 leader_election.cc:288] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759), 834e849fd0174d968aca09501610477d (127.7.132.130:37949)
I20260812 03:47:43.895241 9837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "775ecd746bca4dc785bad2e5c879a3f5" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e73537ce65472c89ead196cd254fba" is_pre_election: true
I20260812 03:47:43.895337 9837 raft_consensus.cc:2454] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 0.
I20260812 03:47:43.895313 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "775ecd746bca4dc785bad2e5c879a3f5" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "834e849fd0174d968aca09501610477d" is_pre_election: true
I20260812 03:47:43.895388 10060 raft_consensus.cc:2454] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 0.
I20260812 03:47:43.895501 10173 leader_election.cc:302] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4737b54ef8564fcc8fe3ba0a5753b7ed, 68e73537ce65472c89ead196cd254fba; no voters:
I20260812 03:47:43.895617 10407 raft_consensus.cc:2781] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:43.895653 10407 raft_consensus.cc:482] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:43.895673 10407 raft_consensus.cc:3037] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.896469 10407 raft_consensus.cc:504] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.896600 10407 leader_election.cc:288] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 election: Requested vote from peers 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759), 834e849fd0174d968aca09501610477d (127.7.132.130:37949)
I20260812 03:47:43.896772 9837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "775ecd746bca4dc785bad2e5c879a3f5" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e73537ce65472c89ead196cd254fba"
I20260812 03:47:43.896807 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "775ecd746bca4dc785bad2e5c879a3f5" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "834e849fd0174d968aca09501610477d"
I20260812 03:47:43.896862 9837 raft_consensus.cc:3037] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.896868 10060 raft_consensus.cc:3037] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.897699 10060 raft_consensus.cc:2454] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 1.
I20260812 03:47:43.897874 10172 leader_election.cc:302] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4737b54ef8564fcc8fe3ba0a5753b7ed, 834e849fd0174d968aca09501610477d; no voters:
I20260812 03:47:43.897962 10407 raft_consensus.cc:2781] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:43.898001 10407 raft_consensus.cc:686] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 LEADER]: Becoming Leader. State: Replica: 4737b54ef8564fcc8fe3ba0a5753b7ed, State: Running, Role: LEADER
I20260812 03:47:43.898053 9837 raft_consensus.cc:2454] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 1.
I20260812 03:47:43.898051 10407 consensus_queue.cc:237] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.898538 9571 catalog_manager.cc:5495] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 4737b54ef8564fcc8fe3ba0a5753b7ed (127.7.132.131). New cstate: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } health_report { overall_health: HEALTHY } } }
Aug 12 03:47:43 dist-test-slave-39l8 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506462, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:43.930632 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.923492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.7.132.190:37233)
0812 03:47:43.923552 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.923554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.923690 (+ 136us) server_negotiation.cc:407] Connection header received
0812 03:47:43.923781 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.923781 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.923819 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.923861 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:43.924339 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.924805 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.925471 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.925644 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.925719 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.928025 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:43.928027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:43.928027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:43.928034 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:43.929421 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.929820 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.929822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.929822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.929879 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:43.930071 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:43.930072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:43.930072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:43.930182 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:43.930251 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.930421 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.930501 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:47:43.930738 10458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.923442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37233 (local address 127.0.0.1:37930)
0812 03:47:43.923595 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.923609 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.923711 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.923890 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.923890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.924209 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:43.924255 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.924257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.924884 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.924885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.925390 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.925392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.925718 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.925719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.925769 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.926343 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:43.926348 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:43.927937 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:43.929523 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.929530 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.929532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.929745 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.929953 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:43.929953 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:43.929954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:43.930013 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:43.930253 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:43.930261 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:43.930306 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.930398 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.930585 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
I20260812 03:47:43.933351 9571 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:37930:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:47:43.933622 9571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:43.935093 10025 tablet_service.cc:1467] Processing CreateTablet for tablet 15229e1f4144407b8f6d131e764f13d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4ead5308d41f4c63b1a439b743f2888c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:47:43.935132 9806 tablet_service.cc:1467] Processing CreateTablet for tablet 15229e1f4144407b8f6d131e764f13d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4ead5308d41f4c63b1a439b743f2888c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:47:43.935166 10261 tablet_service.cc:1467] Processing CreateTablet for tablet 15229e1f4144407b8f6d131e764f13d8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4ead5308d41f4c63b1a439b743f2888c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:47:43.935316 10025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15229e1f4144407b8f6d131e764f13d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.935316 9806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15229e1f4144407b8f6d131e764f13d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.935336 10261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15229e1f4144407b8f6d131e764f13d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:43.938110 10402 tablet_bootstrap.cc:492] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: Bootstrap starting.
I20260812 03:47:43.938128 10399 tablet_bootstrap.cc:492] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Bootstrap starting.
I20260812 03:47:43.938455 10402 tablet_bootstrap.cc:654] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.938570 10399 tablet_bootstrap.cc:654] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.939109 10402 tablet_bootstrap.cc:492] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: No bootstrap required, opened a new log
I20260812 03:47:43.939200 10402 ts_tablet_manager.cc:1397] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:47:43.939399 10400 tablet_bootstrap.cc:492] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: Bootstrap starting.
I20260812 03:47:43.939373 10402 raft_consensus.cc:348] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.939451 10402 raft_consensus.cc:374] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.939481 10402 raft_consensus.cc:729] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 834e849fd0174d968aca09501610477d, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.939524 10402 consensus_queue.cc:260] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.939658 10402 ts_tablet_manager.cc:1428] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:43.939808 10399 tablet_bootstrap.cc:492] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: No bootstrap required, opened a new log
I20260812 03:47:43.939852 10399 ts_tablet_manager.cc:1397] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:47:43.939899 10400 tablet_bootstrap.cc:654] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:43.939898 10408 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.939975 10408 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:43.939990 10399 raft_consensus.cc:348] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.940068 10399 raft_consensus.cc:374] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.940090 10399 raft_consensus.cc:729] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4737b54ef8564fcc8fe3ba0a5753b7ed, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.940135 10399 consensus_queue.cc:260] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.940217 10399 ts_tablet_manager.cc:1428] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.940340 10407 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.940405 10407 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:43.940682 10400 tablet_bootstrap.cc:492] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: No bootstrap required, opened a new log
I20260812 03:47:43.940727 10400 ts_tablet_manager.cc:1397] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:47:43.940860 10400 raft_consensus.cc:348] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.940933 10400 raft_consensus.cc:374] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:43.940953 10400 raft_consensus.cc:729] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68e73537ce65472c89ead196cd254fba, State: Initialized, Role: FOLLOWER
I20260812 03:47:43.941005 10400 consensus_queue.cc:260] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.941097 10400 ts_tablet_manager.cc:1428] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:43.941193 10406 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.941260 10406 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:43.973558 10407 raft_consensus.cc:482] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:43.973668 10407 raft_consensus.cc:504] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.973843 10407 leader_election.cc:288] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759), 834e849fd0174d968aca09501610477d (127.7.132.130:37949)
I20260812 03:47:43.974088 9837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15229e1f4144407b8f6d131e764f13d8" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e73537ce65472c89ead196cd254fba" is_pre_election: true
I20260812 03:47:43.974120 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15229e1f4144407b8f6d131e764f13d8" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "834e849fd0174d968aca09501610477d" is_pre_election: true
I20260812 03:47:43.974203 9837 raft_consensus.cc:2454] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 0.
I20260812 03:47:43.974213 10060 raft_consensus.cc:2454] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 0.
I20260812 03:47:43.974380 10173 leader_election.cc:302] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4737b54ef8564fcc8fe3ba0a5753b7ed, 68e73537ce65472c89ead196cd254fba; no voters:
I20260812 03:47:43.974507 10407 raft_consensus.cc:2781] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:43.974545 10407 raft_consensus.cc:482] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:43.974557 10407 raft_consensus.cc:3037] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.975342 10407 raft_consensus.cc:504] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.975477 10407 leader_election.cc:288] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 election: Requested vote from peers 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759), 834e849fd0174d968aca09501610477d (127.7.132.130:37949)
I20260812 03:47:43.975626 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15229e1f4144407b8f6d131e764f13d8" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "834e849fd0174d968aca09501610477d"
I20260812 03:47:43.975654 9837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15229e1f4144407b8f6d131e764f13d8" candidate_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68e73537ce65472c89ead196cd254fba"
I20260812 03:47:43.975723 10060 raft_consensus.cc:3037] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.975737 9837 raft_consensus.cc:3037] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:43.976326 10060 raft_consensus.cc:2454] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 1.
I20260812 03:47:43.976492 10172 leader_election.cc:302] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4737b54ef8564fcc8fe3ba0a5753b7ed, 834e849fd0174d968aca09501610477d; no voters:
I20260812 03:47:43.976538 9837 raft_consensus.cc:2454] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4737b54ef8564fcc8fe3ba0a5753b7ed in term 1.
I20260812 03:47:43.976577 10407 raft_consensus.cc:2781] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:43.976627 10407 raft_consensus.cc:686] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 LEADER]: Becoming Leader. State: Replica: 4737b54ef8564fcc8fe3ba0a5753b7ed, State: Running, Role: LEADER
I20260812 03:47:43.976701 10407 consensus_queue.cc:237] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } }
I20260812 03:47:43.977011 10407 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: TxnStatusTablet state changed. Reason: New leader 4737b54ef8564fcc8fe3ba0a5753b7ed. Latest consensus state: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.977128 10407 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:47:43.977306 10467 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:47:43.977413 10467 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:47:43.977337 9571 catalog_manager.cc:5495] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 4737b54ef8564fcc8fe3ba0a5753b7ed (127.7.132.131). New cstate: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } health_report { overall_health: HEALTHY } } }
I20260812 03:47:43.989576 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.987068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38544 (local address 127.7.132.131:34607)
0812 03:47:43.987116 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.987119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.987128 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:47:43.987207 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.987208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.987221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.987287 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:47:43.987405 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.987938 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.988804 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.988990 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.989059 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.989272 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:47:43.989307 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:47:43.989384 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.989468 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.989492 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:47:43.989583 10458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.987002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34607 (local address 127.0.0.1:38544)
0812 03:47:43.987068 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.987080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.987133 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.987295 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.987297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.987298 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:47:43.987346 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.987349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.988110 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.988111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.988704 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.988706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.989072 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.989073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.989144 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.989158 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:47:43.989386 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:47:43.989398 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.989463 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.989515 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 03:47:43.990746 9837 raft_consensus.cc:1261] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Refusing update from remote peer 4737b54ef8564fcc8fe3ba0a5753b7ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:43.990738 10060 raft_consensus.cc:1261] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Refusing update from remote peer 4737b54ef8564fcc8fe3ba0a5753b7ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:43.990945 10411 consensus_queue.cc:1035] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:43.991026 10407 consensus_queue.cc:1035] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:43.991554 10406 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: TxnStatusTablet state changed. Reason: New leader 4737b54ef8564fcc8fe3ba0a5753b7ed. Latest consensus state: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.991556 10408 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: TxnStatusTablet state changed. Reason: New leader 4737b54ef8564fcc8fe3ba0a5753b7ed. Latest consensus state: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.991649 10406 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:43.991657 10408 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:43.992578 10407 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.992690 10407 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:47:43.992646 10411 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.992730 10411 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 4737b54ef8564fcc8fe3ba0a5753b7ed: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:47:43.993077 10408 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.993189 10408 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 834e849fd0174d968aca09501610477d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:43.993649 10425 mvcc.cc:205] Tried to move back new op lower bound from 7317530476504158208 to 7317530476449112064. Current Snapshot: MvccSnapshot[applied={T|T < 7317530476504158208}]
I20260812 03:47:43.993758 10406 tablet_replica.cc:439] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 } } peers { permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 } } peers { permanent_uuid: "4737b54ef8564fcc8fe3ba0a5753b7ed" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 34607 } } }
I20260812 03:47:43.993819 10406 tablet_replica.cc:442] T 15229e1f4144407b8f6d131e764f13d8 P 68e73537ce65472c89ead196cd254fba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:43.999421 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.996820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:43499 (local address 127.7.132.131:34607)
0812 03:47:43.996863 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:43.996865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:43.996984 (+ 119us) server_negotiation.cc:407] Connection header received
0812 03:47:43.997045 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:43.997046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:43.997053 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:43.997103 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:47:43.997215 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.997636 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.998659 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.998913 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.998963 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.999184 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:47:43.999212 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:47:43.999258 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:43.999334 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:47:43.999354 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":24}
I20260812 03:47:43.999504 10477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:43.996742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:34607 (local address 127.7.132.131:43499)
0812 03:47:43.996913 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:43.996925 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:43.996990 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:43.997113 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:43.997114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:43.997115 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:47:43.997159 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.997161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.997701 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.997702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.998571 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:43.998574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:43.998980 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:43.998981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:43.999040 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:43.999056 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:47:43.999276 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:47:43.999278 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:47:43.999333 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:47:43.999382 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260812 03:47:44.002133 9837 raft_consensus.cc:1261] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Refusing update from remote peer 4737b54ef8564fcc8fe3ba0a5753b7ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:44.002133 10060 raft_consensus.cc:1261] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Refusing update from remote peer 4737b54ef8564fcc8fe3ba0a5753b7ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:44.002359 10473 consensus_queue.cc:1035] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "68e73537ce65472c89ead196cd254fba" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:44.002467 10407 consensus_queue.cc:1035] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "834e849fd0174d968aca09501610477d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:44.020407 9571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:44.020519 9571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:44.021523 9571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cdb73a044353437191336fd6781c9867: Sending DeleteTablet for 3 replicas of tablet 775ecd746bca4dc785bad2e5c879a3f5
I20260812 03:47:44.021802 10025 tablet_service.cc:1514] Processing DeleteTablet for tablet 775ecd746bca4dc785bad2e5c879a3f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:44 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:33420
I20260812 03:47:44.021862 9806 tablet_service.cc:1514] Processing DeleteTablet for tablet 775ecd746bca4dc785bad2e5c879a3f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:44 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:37046
I20260812 03:47:44.021921 10261 tablet_service.cc:1514] Processing DeleteTablet for tablet 775ecd746bca4dc785bad2e5c879a3f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:44 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:38536
I20260812 03:47:44.021946 10432 tablet_replica.cc:330] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: stopping tablet replica
I20260812 03:47:44.021972 10433 tablet_replica.cc:330] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: stopping tablet replica
I20260812 03:47:44.022017 10434 tablet_replica.cc:330] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: stopping tablet replica
I20260812 03:47:44.022024 10433 raft_consensus.cc:2229] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:44.022025 10432 raft_consensus.cc:2229] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:44.022064 10432 raft_consensus.cc:2258] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:44.022064 10433 raft_consensus.cc:2258] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:44.022073 10434 raft_consensus.cc:2229] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:47:44.022130 10434 raft_consensus.cc:2258] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:44.022439 10434 ts_tablet_manager.cc:1905] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:44.022532 10432 ts_tablet_manager.cc:1905] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:44.023098 10433 ts_tablet_manager.cc:1905] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:44.024093 10432 ts_tablet_manager.cc:1918] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:44.024145 10432 log.cc:1196] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/775ecd746bca4dc785bad2e5c879a3f5
I20260812 03:47:44.024320 10434 ts_tablet_manager.cc:1918] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:44.024356 10432 ts_tablet_manager.cc:1939] T 775ecd746bca4dc785bad2e5c879a3f5 P 834e849fd0174d968aca09501610477d: Deleting consensus metadata
I20260812 03:47:44.024374 10434 log.cc:1196] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/775ecd746bca4dc785bad2e5c879a3f5
I20260812 03:47:44.024672 10434 ts_tablet_manager.cc:1939] T 775ecd746bca4dc785bad2e5c879a3f5 P 4737b54ef8564fcc8fe3ba0a5753b7ed: Deleting consensus metadata
I20260812 03:47:44.024667 9549 catalog_manager.cc:4841] TS 834e849fd0174d968aca09501610477d (127.7.132.130:37949): tablet 775ecd746bca4dc785bad2e5c879a3f5 (table test-table [id=17cedfffe18246b6948a696470a2afa4]) successfully deleted
I20260812 03:47:44.024824 10433 ts_tablet_manager.cc:1918] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:44.024870 10433 log.cc:1196] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/775ecd746bca4dc785bad2e5c879a3f5
I20260812 03:47:44.025022 9554 catalog_manager.cc:4841] TS 4737b54ef8564fcc8fe3ba0a5753b7ed (127.7.132.131:34607): tablet 775ecd746bca4dc785bad2e5c879a3f5 (table test-table [id=17cedfffe18246b6948a696470a2afa4]) successfully deleted
I20260812 03:47:44.025091 10433 ts_tablet_manager.cc:1939] T 775ecd746bca4dc785bad2e5c879a3f5 P 68e73537ce65472c89ead196cd254fba: Deleting consensus metadata
I20260812 03:47:44.025398 9554 catalog_manager.cc:4841] TS 68e73537ce65472c89ead196cd254fba (127.7.132.129:37759): tablet 775ecd746bca4dc785bad2e5c879a3f5 (table test-table [id=17cedfffe18246b6948a696470a2afa4]) successfully deleted
Aug 12 03:47:44 dist-test-slave-39l8 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506463, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:47:44.034813 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:44.025694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.7.132.129:37759)
0812 03:47:44.025740 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:44.025742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:44.025873 (+ 131us) server_negotiation.cc:407] Connection header received
0812 03:47:44.025938 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:44.025939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:44.025948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:44.025990 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:44.026671 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:44.027174 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:44.028187 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:44.028342 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:44.028388 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:44.032009 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:44.032018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:44.032018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:44.032032 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:44.033674 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:44.034055 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:44.034058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:44.034059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:44.034112 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:44.034293 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:44.034293 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:44.034294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:44.034417 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:44.034516 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:44.034617 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:47:44.034689 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 03:47:44.035144 9891 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37056
I20260812 03:47:44.041792 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:44.036127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37938 (local address 127.7.132.190:37233)
0812 03:47:44.036177 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:44.036180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:44.036188 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:47:44.036260 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:44.036260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:44.036270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:44.036324 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:44.036808 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:44.037297 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:44.037959 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:44.038165 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:44.038235 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:44.039396 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:44.039398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:44.039398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:44.039405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:44.040724 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:44.041040 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:44.041041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:44.041041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:44.041101 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:44.041289 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:44.041290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:44.041290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:44.041400 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:44.041495 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:44.041606 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:47:44.041693 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 03:47:44.042012 9571 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37938
I20260812 03:47:44.042783 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9697
I20260812 03:47:44.068430 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9926
I20260812 03:47:44.082566 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10148
I20260812 03:47:44.095424 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9522
2026-08-12T03:47:44Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2567 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:44 dist-test-slave-39l8 krb5kdc[10499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:44 dist-test-slave-39l8 krb5kdc[10499](info): set up 2 sockets
Aug 12 03:47:44 dist-test-slave-39l8 krb5kdc[10499](info): commencing operation
krb5kdc: starting...
W20260812 03:47:45.293289 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:47:45 dist-test-slave-39l8 krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:47:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:47:45.402805 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37781
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:37781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:45.480099 10521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:45.480293 10521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:45.480315 10521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:45.481329 10521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:47:45.481360 10521 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:47:45.481372 10521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:45.481384 10521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:45.481395 10521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:47:45.481405 10521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:47:45.482731 10521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:37781
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:45.482911 10521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:45.483062 10521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:45.487478 10538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:45.487517 10531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:45.487510 10534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:45.487787 10521 server_base.cc:1034] running on GCE node
I20260812 03:47:45.488010 10521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:45.488312 10521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:45.489472 10521 hybrid_clock.cc:648] HybridClock initialized: now 1786506465489459 us; error 41 us; skew 500 ppm
Aug 12 03:47:45 dist-test-slave-39l8 krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506465, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:45.491954 10521 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:47:45.492348 10521 webserver.cc:460] Webserver started at http://127.7.132.190:39019/ using document root <none> and password file <none>
I20260812 03:47:45.492558 10521 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:45.492599 10521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:45.492745 10521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:45.493690 10521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "01eb4e7855b74bb083ec12538f7bd17a"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "39f513a1a5a9c125d3ada9b9d2bd6456"
server_key_iv: "13d36db57d860be036572b9fc583d728"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.494037 10521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "01eb4e7855b74bb083ec12538f7bd17a"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "39f513a1a5a9c125d3ada9b9d2bd6456"
server_key_iv: "13d36db57d860be036572b9fc583d728"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.495376 10521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:47:45.496023 10548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:45.496213 10521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:45.496297 10521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "01eb4e7855b74bb083ec12538f7bd17a"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "39f513a1a5a9c125d3ada9b9d2bd6456"
server_key_iv: "13d36db57d860be036572b9fc583d728"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.496369 10521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:45.513621 10521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:45.515355 10521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:45.515453 10521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:45.519835 10521 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:37781
I20260812 03:47:45.519989 10643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:37781 every 8 connection(s)
I20260812 03:47:45.520238 10521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:47:45.520917 10647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:45.523063 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: Bootstrap starting.
I20260812 03:47:45.523604 10647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:45.523900 10647 log.cc:824] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:45.524559 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: No bootstrap required, opened a new log
I20260812 03:47:45.526484 10647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01eb4e7855b74bb083ec12538f7bd17a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37781 } }
I20260812 03:47:45.526626 10647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:45.526669 10647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01eb4e7855b74bb083ec12538f7bd17a, State: Initialized, Role: FOLLOWER
I20260812 03:47:45.526849 10647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01eb4e7855b74bb083ec12538f7bd17a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37781 } }
I20260812 03:47:45.526937 10647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:47:45.526979 10647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:47:45.527043 10647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:45.527189 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10521
I20260812 03:47:45.527261 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:47:45.527411 7698 external_mini_cluster.cc:1428] Setting key 13df398b8f83eb0ff9878393f8974e7c
I20260812 03:47:45.527691 10647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01eb4e7855b74bb083ec12538f7bd17a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37781 } }
I20260812 03:47:45.527804 10647 leader_election.cc:302] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01eb4e7855b74bb083ec12538f7bd17a; no voters:
I20260812 03:47:45.527974 10647 leader_election.cc:288] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:47:45.528003 10654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:45.528095 10654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [term 1 LEADER]: Becoming Leader. State: Replica: 01eb4e7855b74bb083ec12538f7bd17a, State: Running, Role: LEADER
I20260812 03:47:45.528165 10654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01eb4e7855b74bb083ec12538f7bd17a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37781 } }
I20260812 03:47:45.528208 10647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:47:45.528465 10654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01eb4e7855b74bb083ec12538f7bd17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01eb4e7855b74bb083ec12538f7bd17a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37781 } } }
I20260812 03:47:45.528534 10654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:45.528559 10656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01eb4e7855b74bb083ec12538f7bd17a. Latest consensus state: current_term: 1 leader_uuid: "01eb4e7855b74bb083ec12538f7bd17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01eb4e7855b74bb083ec12538f7bd17a" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37781 } } }
I20260812 03:47:45.528661 10656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:45.528719 10664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:47:45.529197 10664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:47:45.530840 10664 catalog_manager.cc:1347] Generated new cluster ID: 8b8d174761994ac785c29baf28beb626
I20260812 03:47:45.530890 10664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:47:45 dist-test-slave-39l8 krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:45.537309 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.527858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.7.132.190:37781)
0812 03:47:45.528104 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:45.528112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:45.528143 (+ 31us) server_negotiation.cc:407] Connection header received
0812 03:47:45.528245 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:45.528251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:45.528302 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:45.528431 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:45.529488 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.530697 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.531466 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.531881 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.531990 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.534286 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:45.534290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:45.534290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:45.534301 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:45.535862 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.536237 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.536240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.536240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.536341 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.536555 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.536556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.536557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.536778 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:45.536956 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:45.537043 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 03:47:45.537170 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":89,"threads_started":1}
I20260812 03:47:45.556478 10664 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:47:45.556950 10664 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:47:45.563632 10664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: Generated new TSK 0
I20260812 03:47:45.563812 10664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:47:45.593622 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37781
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:45.697628 10691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:45.697846 10691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:45.697886 10691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:45.699188 10691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:45.699241 10691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:45.699292 10691 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:47:45.699319 10691 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:47:45.699361 10691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:47:45.700861 10691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:45.701099 10691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:45.701308 10691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:45.705832 10699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:45.705832 10700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:45.706012 10691 server_base.cc:1034] running on GCE node
W20260812 03:47:45.706097 10702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:45.706269 10691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:45.706485 10691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:45.707614 10691 hybrid_clock.cc:648] HybridClock initialized: now 1786506465707603 us; error 28 us; skew 500 ppm
Aug 12 03:47:45 dist-test-slave-39l8 krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506465, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:45.709955 10691 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:47:45.710366 10691 webserver.cc:460] Webserver started at http://127.7.132.129:44795/ using document root <none> and password file <none>
I20260812 03:47:45.710587 10691 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:45.710628 10691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:45.710803 10691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:45.711809 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "e4610a56ff5b421299760ec1384a2fe4"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "d62f90d5b30107c4e43e15ed4c145e57"
server_key_iv: "fe9f772a30dc65dcb2ee8db3b197084b"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.712175 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "e4610a56ff5b421299760ec1384a2fe4"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "d62f90d5b30107c4e43e15ed4c145e57"
server_key_iv: "fe9f772a30dc65dcb2ee8db3b197084b"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.713449 10691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:47:45.714105 10712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:45.714258 10691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:45.714339 10691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "e4610a56ff5b421299760ec1384a2fe4"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "d62f90d5b30107c4e43e15ed4c145e57"
server_key_iv: "fe9f772a30dc65dcb2ee8db3b197084b"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.714389 10691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:45.728451 10691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:45.730242 10691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:45.730348 10691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:45.730816 10691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:45.730855 10691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:45.730875 10691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:45.730901 10691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:45.737689 10691 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45177
Aug 12 03:47:45 dist-test-slave-39l8 krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506465, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:45.737768 10899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45177 every 8 connection(s)
I20260812 03:47:45.738029 10691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:47:45.738889 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10691
I20260812 03:47:45.738996 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:47:45.739198 7698 external_mini_cluster.cc:1428] Setting key fc05baff992b2deece143fc7663e747d
I20260812 03:47:45.741240 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.731209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:40721 (local address 127.7.132.190:37781)
0812 03:47:45.731274 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:45.731277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:45.731519 (+ 242us) server_negotiation.cc:407] Connection header received
0812 03:47:45.732235 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:45.732236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:45.732251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:45.732294 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:45.733172 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.733916 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.735462 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.735774 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.735846 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.738219 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:45.738221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:45.738221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:45.738228 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:45.739831 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.740251 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.740252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.740253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.740312 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.740577 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.740578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.740578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.740694 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:45.740782 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:45.741052 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 03:47:45.741140 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:47:45.741401 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.731179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37781 (local address 127.7.132.129:40721)
0812 03:47:45.731384 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:45.731434 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:45.732136 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:45.732371 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:45.732375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:45.732777 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:45.733064 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:45.733070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.734025 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.734026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:45.735369 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:45.735373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.735856 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.735857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:45.735995 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.736463 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:45.736474 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:45.738111 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:45.739925 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:45.739933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:45.739935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:45.740171 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:45.740385 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:45.740385 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:45.740386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:45.740503 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:45.740789 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:45.740797 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:45.740925 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 03:47:45.741028 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 03:47:45.741180 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260812 03:47:45.742067 10900 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37781
I20260812 03:47:45.742158 10900 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:45.742323 10900 heartbeater.cc:511] Master 127.7.132.190:37781 requested a full tablet report, sending...
I20260812 03:47:45.742923 10574 ts_manager.cc:194] Registered new tserver with Master: e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177)
I20260812 03:47:45.743753 10574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:40721
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
I20260812 03:47:45.748955 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.743071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:44847 (local address 127.7.132.190:37781)
0812 03:47:45.743116 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:45.743118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:45.743345 (+ 227us) server_negotiation.cc:407] Connection header received
0812 03:47:45.743389 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:45.743390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:45.743399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:45.743454 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:45.743831 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.744305 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.745408 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.745577 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.745633 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.746588 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:45.746589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:45.746589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:45.746595 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:45.747874 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.748278 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.748279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.748279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.748325 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.748505 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.748505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.748506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.748588 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:45.748634 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:45.748784 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 03:47:45.748891 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:47:45.748978 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.743114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37781 (local address 127.7.132.129:44847)
0812 03:47:45.743267 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:45.743278 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:45.743339 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:45.743457 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:45.743458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:45.743682 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:45.743755 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:45.743757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.744403 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.744404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:45.745325 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:45.745328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.745659 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.745660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:45.745743 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.746099 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:45.746105 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:45.746506 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:45.747964 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:45.747965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:45.747965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:45.748185 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:45.748384 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:45.748384 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:45.748384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:45.748448 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:45.748644 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:45.748645 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:45.748691 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 03:47:45.748772 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:47:45.748887 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":78,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:47:45.764887 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37781
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:45.844576 10916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:45.844852 10916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:45.844895 10916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:45.846010 10916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:45.846071 10916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:45.846104 10916 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:47:45.846143 10916 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:47:45.846181 10916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:47:45.847676 10916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:45.847924 10916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:45.848142 10916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:45.852908 10924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:45.853014 10916 server_base.cc:1034] running on GCE node
W20260812 03:47:45.852958 10930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:45.853067 10925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:45.853320 10916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:45.853533 10916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:45.854727 10916 hybrid_clock.cc:648] HybridClock initialized: now 1786506465854682 us; error 37 us; skew 500 ppm
Aug 12 03:47:45 dist-test-slave-39l8 krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506465, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:45.857137 10916 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:47:45.857549 10916 webserver.cc:460] Webserver started at http://127.7.132.130:44693/ using document root <none> and password file <none>
I20260812 03:47:45.857810 10916 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:45.857875 10916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:45.858021 10916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:45.859067 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "3545e7d2fb3c40fea7528414b1fed731"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "e5c60572959b84c92123ab3e1ffbf253"
server_key_iv: "2e49838ee1bf615acb28fbe551c61546"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.859434 10916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "3545e7d2fb3c40fea7528414b1fed731"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "e5c60572959b84c92123ab3e1ffbf253"
server_key_iv: "2e49838ee1bf615acb28fbe551c61546"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.860700 10916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:47:45.861428 10941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:45.861601 10916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:45.861687 10916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "3545e7d2fb3c40fea7528414b1fed731"
format_stamp: "Formatted at 2026-08-12 03:47:45 on dist-test-slave-39l8"
server_key: "e5c60572959b84c92123ab3e1ffbf253"
server_key_iv: "2e49838ee1bf615acb28fbe551c61546"
server_key_version: "encryptionkey@0"
I20260812 03:47:45.861763 10916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:45.876293 10916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:45.878141 10916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:45.878324 10916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:45.878978 10916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:45.879053 10916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:45.879087 10916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:45.879105 10916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:45.886093 10916 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:34395
I20260812 03:47:45.886143 11121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:34395 every 8 connection(s)
Aug 12 03:47:45 dist-test-slave-39l8 krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506465, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:45.886421 10916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:47:45.889530 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.879326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:41731 (local address 127.7.132.190:37781)
0812 03:47:45.879372 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:45.879375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:45.879594 (+ 219us) server_negotiation.cc:407] Connection header received
0812 03:47:45.880124 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:45.880125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:45.880140 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:45.880197 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:45.880850 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.881648 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.883240 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.883536 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.883612 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.886964 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:45.886965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:45.886966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:45.886973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:45.888135 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.888548 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.888549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.888550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.888633 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.888884 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.888885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.888886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.889007 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:45.889094 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:45.889359 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 03:47:45.889441 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:47:45.889729 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.879296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37781 (local address 127.7.132.130:41731)
0812 03:47:45.879454 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:45.879511 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:45.880000 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:45.880264 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:45.880268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:45.880536 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:45.880740 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:45.880745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.881753 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.881755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:45.883148 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:45.883151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.883645 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.883646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:45.883805 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.884885 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:45.884897 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:45.886861 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:45.888236 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:45.888243 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:45.888246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:45.888464 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:45.888703 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:45.888703 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:45.888704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:45.888817 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:45.889086 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:45.889093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:45.889227 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 03:47:45.889334 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:47:45.889487 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260812 03:47:45.889901 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10916
I20260812 03:47:45.890019 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:47:45.890146 11123 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37781
I20260812 03:47:45.890219 7698 external_mini_cluster.cc:1428] Setting key cfec2f58bfb1aee30b09811435d1d879
I20260812 03:47:45.890247 11123 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:45.890445 11123 heartbeater.cc:511] Master 127.7.132.190:37781 requested a full tablet report, sending...
I20260812 03:47:45.890919 10574 ts_manager.cc:194] Registered new tserver with Master: 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130:34395)
I20260812 03:47:45.891433 10574 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:41731
I20260812 03:47:45.896868 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.891383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:50397 (local address 127.7.132.190:37781)
0812 03:47:45.891430 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:45.891432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:45.891440 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:47:45.891519 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:45.891520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:45.891531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:45.891559 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:45.891911 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.892376 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.893103 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.893310 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.893380 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.894250 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:45.894252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:45.894252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:45.894259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:45.895702 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.896087 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.896088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.896088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.896145 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:45.896338 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:45.896339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:45.896339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:45.896482 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:45.896564 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:45.896712 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 03:47:45.896783 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:47:45.896888 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:45.891272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37781 (local address 127.7.132.130:50397)
0812 03:47:45.891388 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:45.891399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:45.891453 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:45.891621 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:45.891622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:45.891773 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:45.891827 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:45.891828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.892474 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.892475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:45.893017 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:45.893018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:45.893380 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:45.893381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:45.893451 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:45.893868 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:45.893872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:45.894130 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:45.895813 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:45.895815 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:45.895815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:45.895994 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:45.896211 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:45.896212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:45.896212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:45.896268 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:45.896560 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:45.896560 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:45.896604 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 03:47:45.896688 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:47:45.896784 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:47:45.917675 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:37781
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:45.999979 11141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:46.000181 11141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:46.000205 11141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:46.001274 11141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:46.001312 11141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:46.001353 11141 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:47:46.001376 11141 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:47:46.001401 11141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:47:46.002827 11141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:37781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:46.003074 11141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:46.003326 11141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:46.008363 11150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:46.008391 11155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:46.008456 11141 server_base.cc:1034] running on GCE node
W20260812 03:47:46.008368 11149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:46.008726 11141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:46.008960 11141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:46.010097 11141 hybrid_clock.cc:648] HybridClock initialized: now 1786506466010085 us; error 47 us; skew 500 ppm
Aug 12 03:47:46 dist-test-slave-39l8 krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506466, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:46.012697 11141 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:47:46.013257 11141 webserver.cc:460] Webserver started at http://127.7.132.131:44543/ using document root <none> and password file <none>
I20260812 03:47:46.013520 11141 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:46.013588 11141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:46.013751 11141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:46.015271 11141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "300e2cdcfd6b434fb198e3f03a278b5a"
format_stamp: "Formatted at 2026-08-12 03:47:46 on dist-test-slave-39l8"
server_key: "5604e298ed798ba7c621662ef6acecbf"
server_key_iv: "b8c843f7645338fe6ddeffc0050595e7"
server_key_version: "encryptionkey@0"
I20260812 03:47:46.015738 11141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "300e2cdcfd6b434fb198e3f03a278b5a"
format_stamp: "Formatted at 2026-08-12 03:47:46 on dist-test-slave-39l8"
server_key: "5604e298ed798ba7c621662ef6acecbf"
server_key_iv: "b8c843f7645338fe6ddeffc0050595e7"
server_key_version: "encryptionkey@0"
I20260812 03:47:46.017582 11141 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:47:46.018508 11164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:46.018718 11141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:46.018822 11141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "300e2cdcfd6b434fb198e3f03a278b5a"
format_stamp: "Formatted at 2026-08-12 03:47:46 on dist-test-slave-39l8"
server_key: "5604e298ed798ba7c621662ef6acecbf"
server_key_iv: "b8c843f7645338fe6ddeffc0050595e7"
server_key_version: "encryptionkey@0"
I20260812 03:47:46.018894 11141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:46.029474 11141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:46.031420 11141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:46.031561 11141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:46.032011 11141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:46.032099 11141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:46.032163 11141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:46.032193 11141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:47:46 dist-test-slave-39l8 krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506466, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:46.039326 11141 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:46859
I20260812 03:47:46.039667 11141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:47:46.039441 11352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:46859 every 8 connection(s)
I20260812 03:47:46.041509 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.032510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:56931 (local address 127.7.132.190:37781)
0812 03:47:46.032560 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.032562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.032839 (+ 277us) server_negotiation.cc:407] Connection header received
0812 03:47:46.033387 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.033388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.033403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.033483 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:46.034344 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.035239 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.036238 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.036451 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.036532 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.038926 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:46.038927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:46.038927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:46.038933 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:46.040531 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:46.040862 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:46.040863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:46.040863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:46.040911 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:46.041062 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:46.041062 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:46.041062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:46.041147 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:46.041200 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.041399 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.041453 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:47:46.041745 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.032513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37781 (local address 127.7.132.131:56931)
0812 03:47:46.032694 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.032727 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.033233 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.033595 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.033600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.033952 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:46.034237 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.034243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.035337 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.035338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.036126 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.036129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.036556 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.036558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.036684 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.037147 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:46.037156 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:46.038831 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:46.040596 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:46.040599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:46.040601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:46.040800 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:46.040949 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:46.040950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:46.040950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:46.041022 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:46.041204 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:46.041208 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:46.041303 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.041387 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.041484 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
I20260812 03:47:46.042223 11353 heartbeater.cc:347] Connected to a master server at 127.7.132.190:37781
I20260812 03:47:46.042303 11353 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:46.042498 11353 heartbeater.cc:511] Master 127.7.132.190:37781 requested a full tablet report, sending...
I20260812 03:47:46.042825 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 11141
I20260812 03:47:46.042922 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:47:46.043094 7698 external_mini_cluster.cc:1428] Setting key 7c2ec8b2c753a18dec0b4c04dc86c695
I20260812 03:47:46.043416 10576 ts_manager.cc:194] Registered new tserver with Master: 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:46.043850 10576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:56931
I20260812 03:47:46.043875 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:47:46.049067 11361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.043215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37781 (local address 127.7.132.131:35901)
0812 03:47:46.043417 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.043427 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.043481 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.043618 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.043619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.043834 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:46.043899 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.043901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.044702 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.044703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.045472 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.045475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.045801 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.045802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.045883 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.046310 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:46.046315 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:46.046557 (+ 242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:46.048070 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:46.048071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:46.048072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:46.048237 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:46.048441 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:46.048442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:46.048442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:46.048498 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:46.048750 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:46.048751 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:46.048793 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.048867 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.048966 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260812 03:47:46.049063 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.043262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:35901 (local address 127.7.132.190:37781)
0812 03:47:46.043300 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.043302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.043470 (+ 168us) server_negotiation.cc:407] Connection header received
0812 03:47:46.043536 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.043536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.043546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.043591 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:46.043982 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.044579 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.045557 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.045728 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.045783 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.046678 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:46.046680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:46.046680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:46.046686 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:46.047973 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:46.048321 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:46.048323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:46.048323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:46.048374 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:46.048577 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:46.048578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:46.048578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:46.048679 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:46.048744 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.048907 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.048971 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 12 03:47:46 dist-test-slave-39l8 krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:47:46 dist-test-slave-39l8 krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:46.067024 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.060548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54906 (local address 127.7.132.190:37781)
0812 03:47:46.060610 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.060612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.060620 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:47:46.060685 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.060686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.060693 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.060740 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:46.061261 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.061667 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.062392 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.062567 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.062637 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.064671 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:46.064672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:46.064672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:46.064678 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:46.065930 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:46.066279 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:46.066282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:46.066282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:46.066345 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:46.066530 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:46.066531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:46.066531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:46.066625 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:46.066753 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.066863 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.066933 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:47:46.068539 10576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:47:46.070001 10576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:46.076244 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.072166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34395 (local address 127.0.0.1:43472)
0812 03:47:46.072475 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.072484 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.072522 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.072849 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.072851 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.072853 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:46.072902 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.072903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.073993 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.073994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.075080 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.075083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.075928 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.075928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.076000 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.076012 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.076111 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.076165 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260812 03:47:46.076252 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.072335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43472 (local address 127.7.132.130:34395)
0812 03:47:46.072637 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.072643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.072655 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:47:46.072697 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.072699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.072715 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.072822 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:46.073081 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.073905 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.075178 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.075849 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.075927 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.075983 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.076131 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.076168 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":90,"threads_started":1}
I20260812 03:47:46.076654 11393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.072347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:46859 (local address 127.0.0.1:59188)
0812 03:47:46.072898 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.072907 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.072951 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.073178 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.073179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.073181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:46.073223 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.073224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.074476 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.074477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.075562 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.075565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.076425 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.076426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.076480 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.076484 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.076534 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.076572 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":85,"threads_started":1}
I20260812 03:47:46.076756 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.072166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45177 (local address 127.0.0.1:38286)
0812 03:47:46.072320 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.072358 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.072418 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.073317 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.073318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.073320 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:46.073363 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.073364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.074510 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.074511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.075444 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.075446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.076570 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.076570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.076620 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.076623 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.076660 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.076695 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":43,"threads_started":1}
I20260812 03:47:46.076954 11023 tablet_service.cc:1467] Processing CreateTablet for tablet a283ee08ba254fcdb4fb076376f168d8 (DEFAULT_TABLE table=test-table [id=b061818c72a545c79ef9bfe8ba9ec56b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:46.077070 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.072333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.7.132.129:45177)
0812 03:47:46.073098 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.073102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.073112 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:47:46.073152 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.073155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.073171 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.073246 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:46.073445 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.074451 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.075523 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.076215 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.076283 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.076912 (+ 629us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.076936 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.076971 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":79,"threads_started":1}
I20260812 03:47:46.077358 11023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a283ee08ba254fcdb4fb076376f168d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:46.077725 10792 tablet_service.cc:1467] Processing CreateTablet for tablet a283ee08ba254fcdb4fb076376f168d8 (DEFAULT_TABLE table=test-table [id=b061818c72a545c79ef9bfe8ba9ec56b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:46.078058 10792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a283ee08ba254fcdb4fb076376f168d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:46.081318 11395 tablet_bootstrap.cc:492] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Bootstrap starting.
I20260812 03:47:46.081640 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.072594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59188 (local address 127.7.132.131:46859)
0812 03:47:46.072756 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.072760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.072976 (+ 216us) server_negotiation.cc:407] Connection header received
0812 03:47:46.073024 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.073026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.073041 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.073112 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:46.073298 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.074374 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.075642 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.076277 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.076331 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.076391 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.081508 (+ 5117us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.081557 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260812 03:47:46.081991 11396 tablet_bootstrap.cc:492] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Bootstrap starting.
I20260812 03:47:46.082299 11243 tablet_service.cc:1467] Processing CreateTablet for tablet a283ee08ba254fcdb4fb076376f168d8 (DEFAULT_TABLE table=test-table [id=b061818c72a545c79ef9bfe8ba9ec56b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:46.082646 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a283ee08ba254fcdb4fb076376f168d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:46.082646 11396 tablet_bootstrap.cc:654] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.082907 11395 tablet_bootstrap.cc:654] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.083030 11396 log.cc:824] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:46.083282 11395 log.cc:824] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:46.083863 11396 tablet_bootstrap.cc:492] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: No bootstrap required, opened a new log
I20260812 03:47:46.083940 11396 ts_tablet_manager.cc:1397] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:47:46.085866 11402 tablet_bootstrap.cc:492] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Bootstrap starting.
I20260812 03:47:46.086046 11396 raft_consensus.cc:348] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.086298 11396 raft_consensus.cc:374] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.086350 11396 raft_consensus.cc:729] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4610a56ff5b421299760ec1384a2fe4, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.086468 11396 consensus_queue.cc:260] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.086774 11396 ts_tablet_manager.cc:1428] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:47:46.086812 10900 heartbeater.cc:503] Master 127.7.132.190:37781 was elected leader, sending a full tablet report...
I20260812 03:47:46.086970 11402 tablet_bootstrap.cc:654] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.087298 11402 log.cc:824] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:46.087389 11395 tablet_bootstrap.cc:492] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: No bootstrap required, opened a new log
I20260812 03:47:46.087461 11395 ts_tablet_manager.cc:1397] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 03:47:46.088147 11402 tablet_bootstrap.cc:492] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: No bootstrap required, opened a new log
I20260812 03:47:46.088202 11402 ts_tablet_manager.cc:1397] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:47:46.089051 11395 raft_consensus.cc:348] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.089165 11395 raft_consensus.cc:374] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.089185 11395 raft_consensus.cc:729] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.089284 11395 consensus_queue.cc:260] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.089557 11395 ts_tablet_manager.cc:1428] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:47:46.089572 11123 heartbeater.cc:503] Master 127.7.132.190:37781 was elected leader, sending a full tablet report...
I20260812 03:47:46.089670 11402 raft_consensus.cc:348] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.089830 11402 raft_consensus.cc:374] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.089882 11402 raft_consensus.cc:729] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300e2cdcfd6b434fb198e3f03a278b5a, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.089998 11402 consensus_queue.cc:260] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.090307 11402 ts_tablet_manager.cc:1428] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:47:46.090417 11353 heartbeater.cc:503] Master 127.7.132.190:37781 was elected leader, sending a full tablet report...
W20260812 03:47:46.137033 11125 tablet.cc:2369] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:47:46.224231 11407 raft_consensus.cc:482] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:46.224390 11407 raft_consensus.cc:504] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.224797 11407 leader_election.cc:288] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:46.227532 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.225003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:46859 (local address 127.7.132.130:43237)
0812 03:47:46.225058 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.225070 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.225111 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.225296 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.225298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.225300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:46.225384 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.225386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.226011 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.226013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.226806 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.226809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.227334 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.227335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.227412 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.227415 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.227455 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.227483 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":27}
I20260812 03:47:46.227551 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.225078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:43237 (local address 127.7.132.131:46859)
0812 03:47:46.225134 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.225136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.225146 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:47:46.225176 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.225177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.225187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.225271 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:46.225453 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.225926 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.226897 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.227251 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.227308 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.227315 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.227478 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.227505 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:47:46.227895 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.225003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45177 (local address 127.7.132.130:47179)
0812 03:47:46.225175 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.225184 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.225227 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.225355 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.225356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.225357 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:47:46.225440 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.225441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.226052 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.226054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.226786 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.226788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.227651 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.227652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.227726 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.227738 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.227786 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.227818 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":13,"thread_start_us":89,"threads_started":1}
I20260812 03:47:46.227890 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.225144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:47179 (local address 127.7.132.129:45177)
0812 03:47:46.225210 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.225212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.225220 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:47:46.225286 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.225287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.225297 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.225350 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:47:46.225507 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.225963 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.226873 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.227255 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.227344 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.227352 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.227793 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.227830 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:47:46.227885 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a283ee08ba254fcdb4fb076376f168d8" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" is_pre_election: true
I20260812 03:47:46.228057 11278 raft_consensus.cc:2454] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:46.228310 10943 leader_election.cc:302] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:46.228408 10822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a283ee08ba254fcdb4fb076376f168d8" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4" is_pre_election: true
I20260812 03:47:46.228452 11407 raft_consensus.cc:2781] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:46.228516 11407 raft_consensus.cc:482] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:46.228562 11407 raft_consensus.cc:3037] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.228547 10822 raft_consensus.cc:2454] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:46.229713 11407 raft_consensus.cc:504] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.229876 11407 leader_election.cc:288] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Requested vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:46.230042 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a283ee08ba254fcdb4fb076376f168d8" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a"
I20260812 03:47:46.230122 11278 raft_consensus.cc:3037] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.230109 10822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a283ee08ba254fcdb4fb076376f168d8" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4"
I20260812 03:47:46.230206 10822 raft_consensus.cc:3037] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.230985 11278 raft_consensus.cc:2454] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:46.231020 10822 raft_consensus.cc:2454] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:46.231186 10943 leader_election.cc:302] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:46.231312 11407 raft_consensus.cc:2781] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:46.231487 11407 raft_consensus.cc:686] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Becoming Leader. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Running, Role: LEADER
I20260812 03:47:46.231585 11407 consensus_queue.cc:237] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.232424 10576 catalog_manager.cc:5495] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } health_report { overall_health: UNKNOWN } } }
W20260812 03:47:46.238792 10901 tablet.cc:2369] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:47:46 dist-test-slave-39l8 krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506465, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:46.285916 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.275337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37781 (local address 127.0.0.1:54908)
0812 03:47:46.275465 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.275474 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.275531 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.275665 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.275665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.275940 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:46.275985 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.275987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.276562 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.276562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.277053 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.277054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.277346 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.277347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.277385 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.282229 (+ 4844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:46.282233 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:46.283554 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:46.285022 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:46.285029 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:46.285030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:46.285196 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:46.285354 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:46.285355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:46.285355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:46.285408 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:46.285594 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:46.285599 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:46.285646 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.285721 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.285807 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260812 03:47:46.285925 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.275400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.7.132.190:37781)
0812 03:47:46.275489 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.275491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.275518 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:47:46.275585 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.275586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.275595 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.275643 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:46.276048 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.276498 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.277133 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.277289 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.277334 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.283637 (+ 6303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:46.283638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:46.283638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:46.283645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:46.284959 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:46.285255 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:46.285256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:46.285257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:46.285301 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:46.285454 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:46.285455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:46.285455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:46.285545 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:46.285591 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.285755 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.285850 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":56}
I20260812 03:47:46.287824 10576 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54908:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:47:46.287995 10576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:46.289418 10792 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8390475e0e47b4bb232fb800223b0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4958038c99a242229b27af4eb5a608ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:47:46.289443 11023 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8390475e0e47b4bb232fb800223b0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4958038c99a242229b27af4eb5a608ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:47:46.289449 11243 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8390475e0e47b4bb232fb800223b0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4958038c99a242229b27af4eb5a608ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:47:46.289614 10792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8390475e0e47b4bb232fb800223b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:46.289618 11023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8390475e0e47b4bb232fb800223b0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:46.289640 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8390475e0e47b4bb232fb800223b0c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 03:47:46.290383 11355 tablet.cc:2369] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:47:46.291082 11402 tablet_bootstrap.cc:492] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: Bootstrap starting.
I20260812 03:47:46.291165 11396 tablet_bootstrap.cc:492] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: Bootstrap starting.
I20260812 03:47:46.291175 11395 tablet_bootstrap.cc:492] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: Bootstrap starting.
I20260812 03:47:46.291548 11395 tablet_bootstrap.cc:654] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.291548 11396 tablet_bootstrap.cc:654] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.291602 11402 tablet_bootstrap.cc:654] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.292359 11395 tablet_bootstrap.cc:492] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: No bootstrap required, opened a new log
I20260812 03:47:46.292379 11396 tablet_bootstrap.cc:492] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: No bootstrap required, opened a new log
I20260812 03:47:46.292415 11395 ts_tablet_manager.cc:1397] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:46.292416 11396 ts_tablet_manager.cc:1397] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:46.292514 11402 tablet_bootstrap.cc:492] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: No bootstrap required, opened a new log
I20260812 03:47:46.292585 11402 ts_tablet_manager.cc:1397] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:47:46.292577 11396 raft_consensus.cc:348] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.292572 11395 raft_consensus.cc:348] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.292645 11396 raft_consensus.cc:374] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.292646 11395 raft_consensus.cc:374] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.292668 11395 raft_consensus.cc:729] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.292668 11396 raft_consensus.cc:729] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4610a56ff5b421299760ec1384a2fe4, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.292713 11395 consensus_queue.cc:260] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.292713 11396 consensus_queue.cc:260] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.292754 11402 raft_consensus.cc:348] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.292809 11395 ts_tablet_manager.cc:1428] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:46.292830 11402 raft_consensus.cc:374] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.292829 11396 ts_tablet_manager.cc:1428] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:46.292852 11402 raft_consensus.cc:729] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300e2cdcfd6b434fb198e3f03a278b5a, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.292897 11402 consensus_queue.cc:260] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.292896 11407 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.292973 11407 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:46.292989 11402 ts_tablet_manager.cc:1428] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:46.292950 11404 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.293025 11404 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:46.293057 11409 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.293145 11409 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:46.604549 11407 raft_consensus.cc:482] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:46.604661 11407 raft_consensus.cc:504] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.604872 11407 leader_election.cc:288] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:46.605116 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8390475e0e47b4bb232fb800223b0c" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" is_pre_election: true
I20260812 03:47:46.605201 11278 raft_consensus.cc:2454] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:46.605135 10822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8390475e0e47b4bb232fb800223b0c" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4" is_pre_election: true
I20260812 03:47:46.605278 10822 raft_consensus.cc:2454] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:46.605377 10943 leader_election.cc:302] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:46.605475 11407 raft_consensus.cc:2781] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:46.605515 11407 raft_consensus.cc:482] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:46.605536 11407 raft_consensus.cc:3037] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.606204 11407 raft_consensus.cc:504] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.606336 11407 leader_election.cc:288] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Requested vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:46.606494 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8390475e0e47b4bb232fb800223b0c" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a"
I20260812 03:47:46.606503 10822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8390475e0e47b4bb232fb800223b0c" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4"
I20260812 03:47:46.606566 11278 raft_consensus.cc:3037] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.606590 10822 raft_consensus.cc:3037] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.607209 11278 raft_consensus.cc:2454] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:46.607231 10822 raft_consensus.cc:2454] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:46.607406 10946 leader_election.cc:302] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3545e7d2fb3c40fea7528414b1fed731, e4610a56ff5b421299760ec1384a2fe4; no voters:
I20260812 03:47:46.607508 11407 raft_consensus.cc:2781] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:46.607551 11407 raft_consensus.cc:686] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Becoming Leader. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Running, Role: LEADER
I20260812 03:47:46.607602 11407 consensus_queue.cc:237] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.607898 11412 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: TxnStatusTablet state changed. Reason: New leader 3545e7d2fb3c40fea7528414b1fed731. Latest consensus state: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.607991 11412 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:47:46.608170 11438 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:47:46.608319 11438 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:47:46.608259 10574 catalog_manager.cc:5495] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } health_report { overall_health: UNKNOWN } } }
I20260812 03:47:46.646189 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.643608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34395 (local address 127.0.0.1:43482)
0812 03:47:46.643821 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.643833 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.643894 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.644076 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.644078 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.644079 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:47:46.644127 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.644130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.644771 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.644772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.645270 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.645272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.645632 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.645633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.645707 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.645722 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:47:46.646023 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:47:46.646034 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.646071 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.646118 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260812 03:47:46.646198 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.643693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.7.132.130:34395)
0812 03:47:46.643805 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.643807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.643869 (+ 62us) server_negotiation.cc:407] Connection header received
0812 03:47:46.643962 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.643963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.643982 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.644069 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:47:46.644187 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.644676 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.645351 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.645542 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.645629 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.645847 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:47:46.645945 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:47:46.646023 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.646085 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.646105 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":68,"threads_started":1}
I20260812 03:47:46.648032 11278 raft_consensus.cc:1261] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:46.648085 10822 raft_consensus.cc:1261] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:46.648375 11407 consensus_queue.cc:1035] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:46.648531 11412 consensus_queue.cc:1035] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:46.649612 11404 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: TxnStatusTablet state changed. Reason: New leader 3545e7d2fb3c40fea7528414b1fed731. Latest consensus state: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.649612 11409 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: TxnStatusTablet state changed. Reason: New leader 3545e7d2fb3c40fea7528414b1fed731. Latest consensus state: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.649699 11404 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:46.649700 11409 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:46.650851 11407 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.650933 11407 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:47:46.650969 11404 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.651041 11404 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P e4610a56ff5b421299760ec1384a2fe4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:46.651031 11407 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.651089 11407 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P 3545e7d2fb3c40fea7528414b1fed731: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:47:46.651332 11446 mvcc.cc:205] Tried to move back new op lower bound from 7317530487387881472 to 7317530487225069568. Current Snapshot: MvccSnapshot[applied={T|T < 7317530487387881472}]
I20260812 03:47:46.651346 11409 tablet_replica.cc:439] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } } }
I20260812 03:47:46.651412 11409 tablet_replica.cc:442] T 7e8390475e0e47b4bb232fb800223b0c P 300e2cdcfd6b434fb198e3f03a278b5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:47:46.651858 11450 mvcc.cc:205] Tried to move back new op lower bound from 7317530487387881472 to 7317530487225069568. Current Snapshot: MvccSnapshot[applied={T|T < 7317530487387881472}]
I20260812 03:47:46.651948 11448 mvcc.cc:205] Tried to move back new op lower bound from 7317530487387881472 to 7317530487225069568. Current Snapshot: MvccSnapshot[applied={T|T < 7317530487387881472}]
I20260812 03:47:46.656090 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.653672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.7.132.130:34395)
0812 03:47:46.653709 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.653711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.653864 (+ 153us) server_negotiation.cc:407] Connection header received
0812 03:47:46.653909 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.653910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.653919 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.653980 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:47:46.654063 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.654713 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.655472 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.655737 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.655802 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.655913 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:47:46.655941 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:47:46.655987 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.656017 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.656034 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:47:46.660775 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.658223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:59951 (local address 127.7.132.130:34395)
0812 03:47:46.658261 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:46.658262 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:46.658398 (+ 136us) server_negotiation.cc:407] Connection header received
0812 03:47:46.658422 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:46.658423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:46.658430 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:46.658495 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:47:46.658614 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.659269 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.659988 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.660277 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.660343 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.660517 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:47:46.660546 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:47:46.660607 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:46.660665 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 03:47:46.660681 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:47:46.660786 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:46.658179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34395 (local address 127.7.132.130:59951)
0812 03:47:46.658301 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:46.658309 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:46.658394 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:46.658519 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:46.658519 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:46.658521 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:47:46.658559 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.658560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.659338 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.659339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.659907 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:46.659908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:46.660361 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:46.660362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:46.660416 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:46.660428 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:47:46.660617 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:47:46.660619 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:47:46.660662 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 03:47:46.660693 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":71,"threads_started":1}
I20260812 03:47:46.662688 11278 raft_consensus.cc:1261] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:46.662722 10822 raft_consensus.cc:1261] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:46.662911 11412 consensus_queue.cc:1035] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:46.662997 11407 consensus_queue.cc:1035] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:46.676339 10574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:46.676440 10574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:46.677142 10574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: Sending DeleteTablet for 3 replicas of tablet a283ee08ba254fcdb4fb076376f168d8
I20260812 03:47:46.677416 10792 tablet_service.cc:1514] Processing DeleteTablet for tablet a283ee08ba254fcdb4fb076376f168d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:46 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:38286
I20260812 03:47:46.677520 11243 tablet_service.cc:1514] Processing DeleteTablet for tablet a283ee08ba254fcdb4fb076376f168d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:46 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59188
I20260812 03:47:46.677609 10574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:47:46.677706 11023 tablet_service.cc:1514] Processing DeleteTablet for tablet a283ee08ba254fcdb4fb076376f168d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:46 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43472
W20260812 03:47:46.677754 10574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:46.677901 11474 tablet_replica.cc:330] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: stopping tablet replica
I20260812 03:47:46.678009 11474 raft_consensus.cc:2229] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:46.678061 11474 raft_consensus.cc:2258] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:46.678211 11475 tablet_replica.cc:330] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: stopping tablet replica
I20260812 03:47:46.678299 11475 raft_consensus.cc:2229] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:47:46.678385 11475 raft_consensus.cc:2258] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:46.678663 11474 ts_tablet_manager.cc:1905] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:46.678802 10792 tablet_service.cc:1467] Processing CreateTablet for tablet e3fa8aa96a23410481047d7750ec9e02 (DEFAULT_TABLE table=test-table [id=950fe8b9e0214e789ddb900bc39f9664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:46.678874 11023 tablet_service.cc:1467] Processing CreateTablet for tablet e3fa8aa96a23410481047d7750ec9e02 (DEFAULT_TABLE table=test-table [id=950fe8b9e0214e789ddb900bc39f9664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:46.678905 10792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3fa8aa96a23410481047d7750ec9e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:46.678984 11023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3fa8aa96a23410481047d7750ec9e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:46.680222 11396 tablet_bootstrap.cc:492] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: Bootstrap starting.
I20260812 03:47:46.680351 11474 ts_tablet_manager.cc:1918] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:46.680402 11395 tablet_bootstrap.cc:492] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: Bootstrap starting.
I20260812 03:47:46.680431 11474 log.cc:1196] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/a283ee08ba254fcdb4fb076376f168d8
I20260812 03:47:46.680677 11396 tablet_bootstrap.cc:654] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.680701 11474 ts_tablet_manager.cc:1939] T a283ee08ba254fcdb4fb076376f168d8 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting consensus metadata
I20260812 03:47:46.680805 11395 tablet_bootstrap.cc:654] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.680362 11243 tablet_service.cc:1467] Processing CreateTablet for tablet e3fa8aa96a23410481047d7750ec9e02 (DEFAULT_TABLE table=test-table [id=950fe8b9e0214e789ddb900bc39f9664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:46.681313 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3fa8aa96a23410481047d7750ec9e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:46.681299 11475 ts_tablet_manager.cc:1905] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:46.681347 11396 tablet_bootstrap.cc:492] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: No bootstrap required, opened a new log
I20260812 03:47:46.681382 11396 ts_tablet_manager.cc:1397] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:46.681550 11396 raft_consensus.cc:348] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.681627 11396 raft_consensus.cc:374] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.681643 11396 raft_consensus.cc:729] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4610a56ff5b421299760ec1384a2fe4, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.681681 11396 consensus_queue.cc:260] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.681753 11396 ts_tablet_manager.cc:1428] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:46.682437 11402 tablet_bootstrap.cc:492] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: Bootstrap starting.
I20260812 03:47:46.682789 11472 tablet_replica.cc:330] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: stopping tablet replica
I20260812 03:47:46.682811 11402 tablet_bootstrap.cc:654] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:46.682838 10550 catalog_manager.cc:4841] TS 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859): tablet a283ee08ba254fcdb4fb076376f168d8 (table test-table [id=b061818c72a545c79ef9bfe8ba9ec56b]) successfully deleted
I20260812 03:47:46.682890 11472 raft_consensus.cc:2229] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:46.682937 11472 raft_consensus.cc:2258] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:46.683274 11472 ts_tablet_manager.cc:1905] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:46.683298 11475 ts_tablet_manager.cc:1918] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:46.683436 11475 log.cc:1196] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/a283ee08ba254fcdb4fb076376f168d8
I20260812 03:47:46.683749 11475 ts_tablet_manager.cc:1939] T a283ee08ba254fcdb4fb076376f168d8 P 3545e7d2fb3c40fea7528414b1fed731: Deleting consensus metadata
I20260812 03:47:46.684176 11402 tablet_bootstrap.cc:492] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: No bootstrap required, opened a new log
I20260812 03:47:46.684253 11402 ts_tablet_manager.cc:1397] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:47:46.684403 11402 raft_consensus.cc:348] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.684473 11402 raft_consensus.cc:374] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.684494 11402 raft_consensus.cc:729] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300e2cdcfd6b434fb198e3f03a278b5a, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.684530 11402 consensus_queue.cc:260] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.684612 11402 ts_tablet_manager.cc:1428] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:46.684779 11472 ts_tablet_manager.cc:1918] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:46.684861 11472 log.cc:1196] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/a283ee08ba254fcdb4fb076376f168d8
I20260812 03:47:46.684860 11395 tablet_bootstrap.cc:492] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: No bootstrap required, opened a new log
I20260812 03:47:46.684899 11395 ts_tablet_manager.cc:1397] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 03:47:46.685050 11395 raft_consensus.cc:348] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.685120 11395 raft_consensus.cc:374] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:46.685140 11395 raft_consensus.cc:729] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Initialized, Role: FOLLOWER
I20260812 03:47:46.685141 11472 ts_tablet_manager.cc:1939] T a283ee08ba254fcdb4fb076376f168d8 P e4610a56ff5b421299760ec1384a2fe4: Deleting consensus metadata
I20260812 03:47:46.685174 11395 consensus_queue.cc:260] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.685263 11395 ts_tablet_manager.cc:1428] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:46.685449 10550 catalog_manager.cc:4841] TS 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130:34395): tablet a283ee08ba254fcdb4fb076376f168d8 (table test-table [id=b061818c72a545c79ef9bfe8ba9ec56b]) successfully deleted
I20260812 03:47:46.685492 10558 catalog_manager.cc:4841] TS e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177): tablet a283ee08ba254fcdb4fb076376f168d8 (table test-table [id=b061818c72a545c79ef9bfe8ba9ec56b]) successfully deleted
I20260812 03:47:46.695012 11467 raft_consensus.cc:482] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:46.695067 11467 raft_consensus.cc:504] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.695211 11467 leader_election.cc:288] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:46.695371 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3fa8aa96a23410481047d7750ec9e02" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" is_pre_election: true
I20260812 03:47:46.695369 10821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3fa8aa96a23410481047d7750ec9e02" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4" is_pre_election: true
I20260812 03:47:46.695425 11278 raft_consensus.cc:2454] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:46.695442 10821 raft_consensus.cc:2454] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:46.695559 10943 leader_election.cc:302] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:46.695647 11467 raft_consensus.cc:2781] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:46.695686 11467 raft_consensus.cc:482] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:46.695716 11467 raft_consensus.cc:3037] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.696242 11467 raft_consensus.cc:504] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.696403 11467 leader_election.cc:288] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Requested vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:46.696516 10821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3fa8aa96a23410481047d7750ec9e02" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4"
I20260812 03:47:46.696578 10821 raft_consensus.cc:3037] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.696579 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3fa8aa96a23410481047d7750ec9e02" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a"
I20260812 03:47:46.696630 11278 raft_consensus.cc:3037] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:46.697212 10821 raft_consensus.cc:2454] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:46.697212 11278 raft_consensus.cc:2454] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:46.697368 10943 leader_election.cc:302] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:46.697489 11467 raft_consensus.cc:2781] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:46.697536 11467 raft_consensus.cc:686] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Becoming Leader. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Running, Role: LEADER
I20260812 03:47:46.697578 11467 consensus_queue.cc:237] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:46.698091 10576 catalog_manager.cc:5495] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } health_report { overall_health: UNKNOWN } } }
I20260812 03:47:46.705479 11278 raft_consensus.cc:1261] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:46.705570 10821 raft_consensus.cc:1261] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:46.705646 11412 consensus_queue.cc:1035] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:46.705752 11467 consensus_queue.cc:1035] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:47:46.710559 11446 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:47:48.227553 10570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:48.227663 10570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:48.228529 10570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: Sending DeleteTablet for 3 replicas of tablet e3fa8aa96a23410481047d7750ec9e02
I20260812 03:47:48.228832 11023 tablet_service.cc:1514] Processing DeleteTablet for tablet e3fa8aa96a23410481047d7750ec9e02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:48 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43472
I20260812 03:47:48.228837 10792 tablet_service.cc:1514] Processing DeleteTablet for tablet e3fa8aa96a23410481047d7750ec9e02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:48 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:38286
I20260812 03:47:48.228881 11243 tablet_service.cc:1514] Processing DeleteTablet for tablet e3fa8aa96a23410481047d7750ec9e02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:48 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59188
I20260812 03:47:48.231769 11529 tablet_replica.cc:330] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: stopping tablet replica
I20260812 03:47:48.231865 11529 raft_consensus.cc:2229] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:48.231921 11529 raft_consensus.cc:2258] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:48.232060 11531 tablet_replica.cc:330] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: stopping tablet replica
I20260812 03:47:48.232127 11531 raft_consensus.cc:2229] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:48.232174 11531 raft_consensus.cc:2258] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:48.232261 11529 ts_tablet_manager.cc:1905] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:48.232286 11526 tablet_replica.cc:330] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: stopping tablet replica
I20260812 03:47:48.232342 11526 raft_consensus.cc:2229] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:47:48.232412 11526 raft_consensus.cc:2258] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:48.232638 11531 ts_tablet_manager.cc:1905] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:48.233989 11529 ts_tablet_manager.cc:1918] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:47:48.234068 11529 log.cc:1196] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/e3fa8aa96a23410481047d7750ec9e02
I20260812 03:47:48.234295 11529 ts_tablet_manager.cc:1939] T e3fa8aa96a23410481047d7750ec9e02 P e4610a56ff5b421299760ec1384a2fe4: Deleting consensus metadata
I20260812 03:47:48.234624 10558 catalog_manager.cc:4841] TS e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177): tablet e3fa8aa96a23410481047d7750ec9e02 (table test-table [id=950fe8b9e0214e789ddb900bc39f9664]) successfully deleted
I20260812 03:47:48.234632 11531 ts_tablet_manager.cc:1918] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:47:48.234733 11531 log.cc:1196] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/e3fa8aa96a23410481047d7750ec9e02
I20260812 03:47:48.235009 11531 ts_tablet_manager.cc:1939] T e3fa8aa96a23410481047d7750ec9e02 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting consensus metadata
I20260812 03:47:48.235083 11526 ts_tablet_manager.cc:1905] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:48.235348 10550 catalog_manager.cc:4841] TS 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859): tablet e3fa8aa96a23410481047d7750ec9e02 (table test-table [id=950fe8b9e0214e789ddb900bc39f9664]) successfully deleted
I20260812 03:47:48.236593 11526 ts_tablet_manager.cc:1918] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:47:48.236666 11526 log.cc:1196] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/e3fa8aa96a23410481047d7750ec9e02
I20260812 03:47:48.236924 11526 ts_tablet_manager.cc:1939] T e3fa8aa96a23410481047d7750ec9e02 P 3545e7d2fb3c40fea7528414b1fed731: Deleting consensus metadata
I20260812 03:47:48.237262 10550 catalog_manager.cc:4841] TS 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130:34395): tablet e3fa8aa96a23410481047d7750ec9e02 (table test-table [id=950fe8b9e0214e789ddb900bc39f9664]) successfully deleted
Aug 12 03:47:48 dist-test-slave-39l8 krb5kdc[10499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506468, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:47:48 dist-test-slave-39l8 krb5kdc[10499](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506468, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:48.254545 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:48.246993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54912 (local address 127.7.132.190:37781)
0812 03:47:48.247150 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:48.247152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:48.247195 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:47:48.247299 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:48.247300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:48.247313 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:48.247386 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:48.247958 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:48.248478 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:48.249222 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:48.249448 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:48.249524 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:48.251858 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:48.251860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:48.251860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:48.251868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:48.253373 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:48.253722 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:48.253724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:48.253724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:48.253791 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:48.253987 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:48.253988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:48.253989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:48.254102 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:48.254206 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:48.254334 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 03:47:48.254423 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260812 03:47:48.256181 10570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:47:48.256362 10570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:48.257761 10792 tablet_service.cc:1467] Processing CreateTablet for tablet 4e6cb4f008f240adbf3c5c0c2588f183 (DEFAULT_TABLE table=test-table [id=b619f0aa83e7401a9073bdeb926d6b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:48.257783 11023 tablet_service.cc:1467] Processing CreateTablet for tablet 4e6cb4f008f240adbf3c5c0c2588f183 (DEFAULT_TABLE table=test-table [id=b619f0aa83e7401a9073bdeb926d6b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:48.257807 11243 tablet_service.cc:1467] Processing CreateTablet for tablet 4e6cb4f008f240adbf3c5c0c2588f183 (DEFAULT_TABLE table=test-table [id=b619f0aa83e7401a9073bdeb926d6b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:48.257905 11023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e6cb4f008f240adbf3c5c0c2588f183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:48.257903 10792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e6cb4f008f240adbf3c5c0c2588f183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:48.257915 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e6cb4f008f240adbf3c5c0c2588f183. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:48.259114 11547 tablet_bootstrap.cc:492] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: Bootstrap starting.
I20260812 03:47:48.259519 11547 tablet_bootstrap.cc:654] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:48.259524 11553 tablet_bootstrap.cc:492] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: Bootstrap starting.
I20260812 03:47:48.259910 11553 tablet_bootstrap.cc:654] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:48.260156 11547 tablet_bootstrap.cc:492] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: No bootstrap required, opened a new log
I20260812 03:47:48.260236 11547 ts_tablet_manager.cc:1397] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:48.260255 11551 tablet_bootstrap.cc:492] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: Bootstrap starting.
I20260812 03:47:48.260358 11547 raft_consensus.cc:348] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.260440 11547 raft_consensus.cc:374] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:48.260465 11547 raft_consensus.cc:729] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Initialized, Role: FOLLOWER
I20260812 03:47:48.260533 11547 consensus_queue.cc:260] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.260552 11553 tablet_bootstrap.cc:492] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: No bootstrap required, opened a new log
I20260812 03:47:48.260598 11553 ts_tablet_manager.cc:1397] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:48.260603 11551 tablet_bootstrap.cc:654] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:48.260784 11547 ts_tablet_manager.cc:1428] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:48.260782 11553 raft_consensus.cc:348] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.260859 11553 raft_consensus.cc:374] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:48.260881 11553 raft_consensus.cc:729] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300e2cdcfd6b434fb198e3f03a278b5a, State: Initialized, Role: FOLLOWER
I20260812 03:47:48.260922 11553 consensus_queue.cc:260] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.261075 11553 ts_tablet_manager.cc:1428] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:48.261395 11551 tablet_bootstrap.cc:492] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: No bootstrap required, opened a new log
I20260812 03:47:48.261444 11551 ts_tablet_manager.cc:1397] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:48.261615 11551 raft_consensus.cc:348] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.261698 11551 raft_consensus.cc:374] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:48.261727 11551 raft_consensus.cc:729] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4610a56ff5b421299760ec1384a2fe4, State: Initialized, Role: FOLLOWER
I20260812 03:47:48.261768 11551 consensus_queue.cc:260] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.261931 11551 ts_tablet_manager.cc:1428] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:48.606022 11412 raft_consensus.cc:482] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:48.606133 11412 raft_consensus.cc:504] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.606323 11412 leader_election.cc:288] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:48.606577 10821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e6cb4f008f240adbf3c5c0c2588f183" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4" is_pre_election: true
I20260812 03:47:48.606616 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e6cb4f008f240adbf3c5c0c2588f183" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" is_pre_election: true
I20260812 03:47:48.606731 10821 raft_consensus.cc:2454] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:48.606748 11278 raft_consensus.cc:2454] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:48.606920 10943 leader_election.cc:302] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:48.607074 11412 raft_consensus.cc:2781] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:48.607132 11412 raft_consensus.cc:482] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:48.607154 11412 raft_consensus.cc:3037] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:48.607899 11412 raft_consensus.cc:504] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.608055 11412 leader_election.cc:288] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Requested vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:48.608204 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e6cb4f008f240adbf3c5c0c2588f183" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a"
I20260812 03:47:48.608289 11278 raft_consensus.cc:3037] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:48.608250 10821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e6cb4f008f240adbf3c5c0c2588f183" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4"
I20260812 03:47:48.608338 10821 raft_consensus.cc:3037] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:48.608912 11278 raft_consensus.cc:2454] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:48.609050 10821 raft_consensus.cc:2454] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:48.609107 10943 leader_election.cc:302] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:48.609217 11412 raft_consensus.cc:2781] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:48.609284 11412 raft_consensus.cc:686] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Becoming Leader. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Running, Role: LEADER
I20260812 03:47:48.609349 11412 consensus_queue.cc:237] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.610018 10576 catalog_manager.cc:5495] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } health_report { overall_health: UNKNOWN } } }
I20260812 03:47:48.703220 11565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:48.701167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.7.132.130:34395)
0812 03:47:48.701315 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:48.701316 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:48.701322 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:47:48.701341 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:48.701341 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:48.701350 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:48.701400 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:47:48.701514 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:48.701966 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:48.702605 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:48.702857 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:48.702922 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:48.703042 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:47:48.703069 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:47:48.703100 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:48.703136 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 03:47:48.703159 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260812 03:47:48.705307 10821 raft_consensus.cc:1261] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:48.705317 11278 raft_consensus.cc:1261] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:48.705543 11502 consensus_queue.cc:1035] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:48.705709 11568 consensus_queue.cc:1035] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:48.715245 10574 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:48.715346 10574 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:48.715960 10574 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: Sending DeleteTablet for 3 replicas of tablet 4e6cb4f008f240adbf3c5c0c2588f183
I20260812 03:47:48.716216 11023 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e6cb4f008f240adbf3c5c0c2588f183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:48 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43472
I20260812 03:47:48.716213 11243 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e6cb4f008f240adbf3c5c0c2588f183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:48 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59188
I20260812 03:47:48.716212 10792 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e6cb4f008f240adbf3c5c0c2588f183 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:48 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:38286
I20260812 03:47:48.716307 11529 tablet_replica.cc:330] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: stopping tablet replica
I20260812 03:47:48.716312 11531 tablet_replica.cc:330] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: stopping tablet replica
I20260812 03:47:48.716308 11526 tablet_replica.cc:330] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: stopping tablet replica
I20260812 03:47:48.716348 11529 raft_consensus.cc:2229] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:48.716363 11531 raft_consensus.cc:2229] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:48.716369 11526 raft_consensus.cc:2229] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:47:48.716384 11529 raft_consensus.cc:2258] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:48.716400 11531 raft_consensus.cc:2258] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:48.716415 11526 raft_consensus.cc:2258] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:48.716702 11529 ts_tablet_manager.cc:1905] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:48.716718 10574 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:47:48.716804 11526 ts_tablet_manager.cc:1905] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:47:48.716848 10574 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:47:48.718295 11529 ts_tablet_manager.cc:1918] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:48.718346 11529 log.cc:1196] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/4e6cb4f008f240adbf3c5c0c2588f183
I20260812 03:47:48.718327 11526 ts_tablet_manager.cc:1918] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:48.718407 11526 log.cc:1196] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/4e6cb4f008f240adbf3c5c0c2588f183
I20260812 03:47:48.718550 11529 ts_tablet_manager.cc:1939] T 4e6cb4f008f240adbf3c5c0c2588f183 P e4610a56ff5b421299760ec1384a2fe4: Deleting consensus metadata
I20260812 03:47:48.718664 11526 ts_tablet_manager.cc:1939] T 4e6cb4f008f240adbf3c5c0c2588f183 P 3545e7d2fb3c40fea7528414b1fed731: Deleting consensus metadata
I20260812 03:47:48.718896 10558 catalog_manager.cc:4841] TS e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177): tablet 4e6cb4f008f240adbf3c5c0c2588f183 (table test-table [id=b619f0aa83e7401a9073bdeb926d6b59]) successfully deleted
I20260812 03:47:48.718955 11531 ts_tablet_manager.cc:1905] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:48.719163 10550 catalog_manager.cc:4841] TS 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130:34395): tablet 4e6cb4f008f240adbf3c5c0c2588f183 (table test-table [id=b619f0aa83e7401a9073bdeb926d6b59]) successfully deleted
I20260812 03:47:48.719697 10792 tablet_service.cc:1467] Processing CreateTablet for tablet bd64d95d085b4ee5a0f7c7fde14a78a8 (DEFAULT_TABLE table=test-table [id=1d53de5efa4f4df1847a833a2372012a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:48.719774 11243 tablet_service.cc:1467] Processing CreateTablet for tablet bd64d95d085b4ee5a0f7c7fde14a78a8 (DEFAULT_TABLE table=test-table [id=1d53de5efa4f4df1847a833a2372012a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:48.719779 11023 tablet_service.cc:1467] Processing CreateTablet for tablet bd64d95d085b4ee5a0f7c7fde14a78a8 (DEFAULT_TABLE table=test-table [id=1d53de5efa4f4df1847a833a2372012a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:47:48.719810 10792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd64d95d085b4ee5a0f7c7fde14a78a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:48.719866 11243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd64d95d085b4ee5a0f7c7fde14a78a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:48.719883 11023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd64d95d085b4ee5a0f7c7fde14a78a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:48.720705 11531 ts_tablet_manager.cc:1918] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:47:48.720763 11531 log.cc:1196] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/4e6cb4f008f240adbf3c5c0c2588f183
I20260812 03:47:48.720966 11531 ts_tablet_manager.cc:1939] T 4e6cb4f008f240adbf3c5c0c2588f183 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting consensus metadata
I20260812 03:47:48.720971 11551 tablet_bootstrap.cc:492] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4: Bootstrap starting.
I20260812 03:47:48.721223 11547 tablet_bootstrap.cc:492] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: Bootstrap starting.
I20260812 03:47:48.721354 10550 catalog_manager.cc:4841] TS 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859): tablet 4e6cb4f008f240adbf3c5c0c2588f183 (table test-table [id=b619f0aa83e7401a9073bdeb926d6b59]) successfully deleted
I20260812 03:47:48.721383 11551 tablet_bootstrap.cc:654] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:48.721499 11547 tablet_bootstrap.cc:654] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:48.721715 11553 tablet_bootstrap.cc:492] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: Bootstrap starting.
I20260812 03:47:48.722057 11551 tablet_bootstrap.cc:492] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4: No bootstrap required, opened a new log
I20260812 03:47:48.722121 11553 tablet_bootstrap.cc:654] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:48.722118 11551 ts_tablet_manager.cc:1397] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:48.722267 11551 raft_consensus.cc:348] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.722357 11551 raft_consensus.cc:374] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:48.722378 11551 raft_consensus.cc:729] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4610a56ff5b421299760ec1384a2fe4, State: Initialized, Role: FOLLOWER
I20260812 03:47:48.722414 11551 consensus_queue.cc:260] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.722522 11551 ts_tablet_manager.cc:1428] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:48.723112 11553 tablet_bootstrap.cc:492] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: No bootstrap required, opened a new log
I20260812 03:47:48.723181 11553 ts_tablet_manager.cc:1397] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:48.723251 11547 tablet_bootstrap.cc:492] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: No bootstrap required, opened a new log
I20260812 03:47:48.723302 11547 ts_tablet_manager.cc:1397] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260812 03:47:48.723341 11553 raft_consensus.cc:348] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.723409 11553 raft_consensus.cc:374] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:48.723412 11547 raft_consensus.cc:348] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.723455 11553 raft_consensus.cc:729] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300e2cdcfd6b434fb198e3f03a278b5a, State: Initialized, Role: FOLLOWER
I20260812 03:47:48.723470 11547 raft_consensus.cc:374] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:48.723492 11547 raft_consensus.cc:729] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Initialized, Role: FOLLOWER
I20260812 03:47:48.723508 11553 consensus_queue.cc:260] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.723565 11547 consensus_queue.cc:260] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:48.723621 11553 ts_tablet_manager.cc:1428] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:48.723665 11547 ts_tablet_manager.cc:1428] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:49.011695 11494 raft_consensus.cc:482] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:47:49.011839 11494 raft_consensus.cc:504] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:49.012070 11494 leader_election.cc:288] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:49.012267 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd64d95d085b4ee5a0f7c7fde14a78a8" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" is_pre_election: true
I20260812 03:47:49.012267 10821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd64d95d085b4ee5a0f7c7fde14a78a8" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4" is_pre_election: true
I20260812 03:47:49.012358 11278 raft_consensus.cc:2454] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:49.012391 10821 raft_consensus.cc:2454] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 0.
I20260812 03:47:49.012540 10943 leader_election.cc:302] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:49.012660 11494 raft_consensus.cc:2781] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:47:49.012712 11494 raft_consensus.cc:482] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:47:49.012742 11494 raft_consensus.cc:3037] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:49.013485 11494 raft_consensus.cc:504] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:49.013616 11494 leader_election.cc:288] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Requested vote from peers e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177), 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859)
I20260812 03:47:49.013768 11278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd64d95d085b4ee5a0f7c7fde14a78a8" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "300e2cdcfd6b434fb198e3f03a278b5a"
I20260812 03:47:49.013773 10821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd64d95d085b4ee5a0f7c7fde14a78a8" candidate_uuid: "3545e7d2fb3c40fea7528414b1fed731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4610a56ff5b421299760ec1384a2fe4"
I20260812 03:47:49.013862 11278 raft_consensus.cc:3037] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:49.013859 10821 raft_consensus.cc:3037] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:49.014442 11278 raft_consensus.cc:2454] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:49.014451 10821 raft_consensus.cc:2454] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3545e7d2fb3c40fea7528414b1fed731 in term 1.
I20260812 03:47:49.014604 10943 leader_election.cc:302] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 300e2cdcfd6b434fb198e3f03a278b5a, 3545e7d2fb3c40fea7528414b1fed731; no voters:
I20260812 03:47:49.014729 11494 raft_consensus.cc:2781] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:49.014781 11494 raft_consensus.cc:686] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Becoming Leader. State: Replica: 3545e7d2fb3c40fea7528414b1fed731, State: Running, Role: LEADER
I20260812 03:47:49.014865 11494 consensus_queue.cc:237] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } }
I20260812 03:47:49.015473 10574 catalog_manager.cc:5495] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "3545e7d2fb3c40fea7528414b1fed731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3545e7d2fb3c40fea7528414b1fed731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 } health_report { overall_health: UNKNOWN } } }
I20260812 03:47:49.078181 11278 raft_consensus.cc:1261] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:49.078248 10821 raft_consensus.cc:1261] T bd64d95d085b4ee5a0f7c7fde14a78a8 P e4610a56ff5b421299760ec1384a2fe4 [term 1 FOLLOWER]: Refusing update from remote peer 3545e7d2fb3c40fea7528414b1fed731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:47:49.078491 11502 consensus_queue.cc:1035] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "300e2cdcfd6b434fb198e3f03a278b5a" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 46859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:47:49.078603 11568 consensus_queue.cc:1035] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4610a56ff5b421299760ec1384a2fe4" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:47:49.082945 11491 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:47:50.601085 10570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:50.601205 10570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:47:50.601958 10570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01eb4e7855b74bb083ec12538f7bd17a: Sending DeleteTablet for 3 replicas of tablet bd64d95d085b4ee5a0f7c7fde14a78a8
I20260812 03:47:50.602247 11023 tablet_service.cc:1514] Processing DeleteTablet for tablet bd64d95d085b4ee5a0f7c7fde14a78a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:50 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43472
I20260812 03:47:50.602247 10792 tablet_service.cc:1514] Processing DeleteTablet for tablet bd64d95d085b4ee5a0f7c7fde14a78a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:50 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:38286
I20260812 03:47:50.602398 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10691
I20260812 03:47:50.602424 11243 tablet_service.cc:1514] Processing DeleteTablet for tablet bd64d95d085b4ee5a0f7c7fde14a78a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:47:50 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59188
I20260812 03:47:50.606662 11625 tablet_replica.cc:330] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: stopping tablet replica
I20260812 03:47:50.606750 11625 raft_consensus.cc:2229] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:47:50.606793 11625 raft_consensus.cc:2258] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:50.607012 11625 ts_tablet_manager.cc:1905] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:50.607941 11623 tablet_replica.cc:330] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: stopping tablet replica
I20260812 03:47:50.608078 11623 raft_consensus.cc:2229] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:47:50.608263 11623 raft_consensus.cc:2258] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:47:50.608563 11623 ts_tablet_manager.cc:1905] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:47:50.608881 11625 ts_tablet_manager.cc:1918] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:47:50.608946 11625 log.cc:1196] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/bd64d95d085b4ee5a0f7c7fde14a78a8
I20260812 03:47:50.609174 11625 ts_tablet_manager.cc:1939] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 300e2cdcfd6b434fb198e3f03a278b5a: Deleting consensus metadata
I20260812 03:47:50.609547 10550 catalog_manager.cc:4841] TS 300e2cdcfd6b434fb198e3f03a278b5a (127.7.132.131:46859): tablet bd64d95d085b4ee5a0f7c7fde14a78a8 (table test-table [id=1d53de5efa4f4df1847a833a2372012a]) successfully deleted
I20260812 03:47:50.610148 11623 ts_tablet_manager.cc:1918] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:47:50.610198 11623 log.cc:1196] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/bd64d95d085b4ee5a0f7c7fde14a78a8
I20260812 03:47:50.610370 11623 ts_tablet_manager.cc:1939] T bd64d95d085b4ee5a0f7c7fde14a78a8 P 3545e7d2fb3c40fea7528414b1fed731: Deleting consensus metadata
I20260812 03:47:50.610816 10550 catalog_manager.cc:4841] TS 3545e7d2fb3c40fea7528414b1fed731 (127.7.132.130:34395): tablet bd64d95d085b4ee5a0f7c7fde14a78a8 (table test-table [id=1d53de5efa4f4df1847a833a2372012a]) successfully deleted
W20260812 03:47:50.615077 10558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:45177 (error 108)
I20260812 03:47:50.615319 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10916
I20260812 03:47:50.620012 11627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:50.615443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45177 (local address 127.0.0.1:38302)
0812 03:47:50.619894 (+ 4451us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:50.619959 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4418,"thread_start_us":61,"threads_started":1}
W20260812 03:47:50.620185 10558 catalog_manager.cc:4584] TS e4610a56ff5b421299760ec1384a2fe4 (127.7.132.129:45177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd64d95d085b4ee5a0f7c7fde14a78a8: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45177: connect: Connection refused (error 111)
I20260812 03:47:50.625931 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 11141
I20260812 03:47:50.635257 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10521
2026-08-12T03:47:50Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6536 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:50 dist-test-slave-39l8 krb5kdc[11631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:50 dist-test-slave-39l8 krb5kdc[11631](info): set up 2 sockets
Aug 12 03:47:50 dist-test-slave-39l8 krb5kdc[11631](info): commencing operation
krb5kdc: starting...
W20260812 03:47:51.821841 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:47:51 dist-test-slave-39l8 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:47:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:47:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:47:51.932917 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46109
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:52.013183 11653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:52.013473 11653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:52.013510 11653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:52.015616 11653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:47:52.015684 11653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:52.015719 11653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:52.015749 11653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:47:52.015785 11653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:47:52.018014 11653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:52.018250 11653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:52.018504 11653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:52.023190 11659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:52.023211 11662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:52.023247 11660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:52.023602 11653 server_base.cc:1034] running on GCE node
I20260812 03:47:52.023800 11653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:52.024019 11653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:52.025149 11653 hybrid_clock.cc:648] HybridClock initialized: now 1786506472025135 us; error 29 us; skew 500 ppm
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:52.030114 11653 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:47:52.030580 11653 webserver.cc:460] Webserver started at http://127.7.132.190:43619/ using document root <none> and password file <none>
I20260812 03:47:52.030865 11653 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:52.030936 11653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:52.031093 11653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:52.032136 11653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "11dda5f23f5547f88a86407668847222"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "0352130f4295c93ca71e3d3a8e8d1327"
server_key_iv: "e2d6f18e2dadc13d30fe6651a012f59c"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.032503 11653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "11dda5f23f5547f88a86407668847222"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "0352130f4295c93ca71e3d3a8e8d1327"
server_key_iv: "e2d6f18e2dadc13d30fe6651a012f59c"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.033842 11653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:47:52.034523 11675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.034686 11653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:52.034811 11653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "11dda5f23f5547f88a86407668847222"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "0352130f4295c93ca71e3d3a8e8d1327"
server_key_iv: "e2d6f18e2dadc13d30fe6651a012f59c"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.034895 11653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:52.051723 11653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:52.053879 11653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:52.054033 11653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:52.057890 11653 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46109
I20260812 03:47:52.057922 11758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46109 every 8 connection(s)
I20260812 03:47:52.058248 11653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:47:52.059149 11759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:52.061295 11759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222: Bootstrap starting.
I20260812 03:47:52.061775 11759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:52.062005 11759 log.cc:824] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:52.062672 11759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222: No bootstrap required, opened a new log
I20260812 03:47:52.064222 11759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dda5f23f5547f88a86407668847222" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46109 } }
I20260812 03:47:52.064368 11759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:52.064388 11759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11dda5f23f5547f88a86407668847222, State: Initialized, Role: FOLLOWER
I20260812 03:47:52.064482 11759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dda5f23f5547f88a86407668847222" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46109 } }
I20260812 03:47:52.064532 11759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:47:52.064551 11759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:47:52.064576 11759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:52.065032 11759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dda5f23f5547f88a86407668847222" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46109 } }
I20260812 03:47:52.065111 11759 leader_election.cc:302] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11dda5f23f5547f88a86407668847222; no voters:
I20260812 03:47:52.065229 11759 leader_election.cc:288] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:47:52.065294 11765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:52.065471 11765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [term 1 LEADER]: Becoming Leader. State: Replica: 11dda5f23f5547f88a86407668847222, State: Running, Role: LEADER
I20260812 03:47:52.065478 11759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:47:52.065568 11765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dda5f23f5547f88a86407668847222" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46109 } }
I20260812 03:47:52.065933 11767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11dda5f23f5547f88a86407668847222. Latest consensus state: current_term: 1 leader_uuid: "11dda5f23f5547f88a86407668847222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dda5f23f5547f88a86407668847222" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46109 } } }
I20260812 03:47:52.066022 11767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:52.066244 11766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11dda5f23f5547f88a86407668847222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dda5f23f5547f88a86407668847222" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46109 } } }
I20260812 03:47:52.066391 11766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:52.066560 11775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:47:52.067154 11775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:47:52.067319 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 11653
I20260812 03:47:52.067414 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:47:52.067606 7698 external_mini_cluster.cc:1428] Setting key 2978392568bfe3168d341710a4a7390d
I20260812 03:47:52.068620 11775 catalog_manager.cc:1347] Generated new cluster ID: 280d83ad6a8b4ab2a32f7538089a6e4e
I20260812 03:47:52.068717 11775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:52.077085 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:52.068232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34416 (local address 127.7.132.190:46109)
0812 03:47:52.068646 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:52.068653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:52.068682 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:47:52.068773 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:52.068778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:52.068814 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:52.068953 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:52.069512 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.070455 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.071268 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.071687 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.071816 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:52.073994 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:52.073997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:52.073998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:52.074019 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:52.075618 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:52.075987 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:52.075990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:52.075991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:52.076088 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:52.076296 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:52.076297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:52.076298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:52.076512 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:52.076678 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:52.076778 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:47:52.076913 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":95,"threads_started":1}
I20260812 03:47:52.088740 11775 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:47:52.089273 11775 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:47:52.095755 11775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11dda5f23f5547f88a86407668847222: Generated new TSK 0
I20260812 03:47:52.095911 11775 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:47:52.127133 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46109
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:52.226356 11800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:52.226528 11800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:52.226549 11800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:52.227632 11800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:52.227669 11800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:52.227690 11800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:47:52.228885 11800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:52.229054 11800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:52.229216 11800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:52.233937 11815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:52.233989 11809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:52.233944 11808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:52.234248 11800 server_base.cc:1034] running on GCE node
I20260812 03:47:52.234486 11800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:52.234755 11800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:52.235910 11800 hybrid_clock.cc:648] HybridClock initialized: now 1786506472235893 us; error 34 us; skew 500 ppm
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:52.238404 11800 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:47:52.238915 11800 webserver.cc:460] Webserver started at http://127.7.132.129:35903/ using document root <none> and password file <none>
I20260812 03:47:52.239166 11800 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:52.239241 11800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:52.239373 11800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:52.240424 11800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "5d1c14effe1946abbed8ae63bfa36104"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "5ab813d059c67850d0e870adee7ce883"
server_key_iv: "11013a22478a6ee00fb95b5118a26df6"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.240789 11800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "5d1c14effe1946abbed8ae63bfa36104"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "5ab813d059c67850d0e870adee7ce883"
server_key_iv: "11013a22478a6ee00fb95b5118a26df6"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.242046 11800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:47:52.242687 11825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.242924 11800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.243023 11800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "5d1c14effe1946abbed8ae63bfa36104"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "5ab813d059c67850d0e870adee7ce883"
server_key_iv: "11013a22478a6ee00fb95b5118a26df6"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.243108 11800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:52.260691 11800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:52.262518 11800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:52.262676 11800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:52.262931 11800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:52.263269 11800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:52.263301 11800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.263322 11800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:52.263370 11800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.269158 11800 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:41281
I20260812 03:47:52.269207 12014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:41281 every 8 connection(s)
I20260812 03:47:52.269479 11800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:47:52.272354 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 11800
I20260812 03:47:52.272465 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:47:52.272643 7698 external_mini_cluster.cc:1428] Setting key 709239fa73ec527afac25a87c456c2a9
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:52.278481 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:52.270161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:47517 (local address 127.7.132.190:46109)
0812 03:47:52.270223 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:52.270225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:52.270447 (+ 222us) server_negotiation.cc:407] Connection header received
0812 03:47:52.270988 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:52.270989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:52.270999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:52.271061 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:52.271866 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.272383 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.273635 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.273833 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.273908 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:52.276030 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:52.276031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:52.276032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:52.276039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:52.277214 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:52.277591 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:52.277592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:52.277592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:52.277648 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:52.277883 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:52.277884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:52.277884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:52.277987 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:52.278057 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:52.278311 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 03:47:52.278399 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:47:52.278625 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:52.270138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46109 (local address 127.7.132.129:47517)
0812 03:47:52.270312 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:52.270353 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:52.270903 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:52.271128 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:52.271136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:52.271626 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:52.271795 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:52.271800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.272482 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.272483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:52.273556 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:52.273558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.273920 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.273921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:52.274034 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:52.274576 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:52.274584 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:52.275943 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:52.277303 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:52.277310 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:52.277312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:52.277507 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:52.277711 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:52.277711 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:52.277712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:52.277813 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:52.278049 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:52.278056 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:52.278181 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 03:47:52.278288 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:47:52.278424 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260812 03:47:52.279098 12015 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46109
I20260812 03:47:52.279232 12015 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:52.279423 12015 heartbeater.cc:511] Master 127.7.132.190:46109 requested a full tablet report, sending...
I20260812 03:47:52.279983 11706 ts_manager.cc:194] Registered new tserver with Master: 5d1c14effe1946abbed8ae63bfa36104 (127.7.132.129:41281)
I20260812 03:47:52.280586 11706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:47517
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:47:52.295607 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46109
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:52.373879 12024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:52.374055 12024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:52.374075 12024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:52.375118 12024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:52.375159 12024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:52.375182 12024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:47:52.376436 12024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:52.376610 12024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:52.376760 12024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:52.381413 12033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:52.381443 12034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:52.381461 12036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:52.381580 12024 server_base.cc:1034] running on GCE node
I20260812 03:47:52.381875 12024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:52.382102 12024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:52.383253 12024 hybrid_clock.cc:648] HybridClock initialized: now 1786506472383241 us; error 42 us; skew 500 ppm
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:52.385562 12024 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:47:52.386009 12024 webserver.cc:460] Webserver started at http://127.7.132.130:36455/ using document root <none> and password file <none>
I20260812 03:47:52.386229 12024 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:52.386292 12024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:52.386461 12024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:52.387486 12024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "10fb29450da54f3f8cfff7473e333b64"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "2e84ae34a138a1ce2ae40e3f916e61dd"
server_key_iv: "c01bc4ede66f0cb6ea4a6cb9bf29d311"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.387859 12024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "10fb29450da54f3f8cfff7473e333b64"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "2e84ae34a138a1ce2ae40e3f916e61dd"
server_key_iv: "c01bc4ede66f0cb6ea4a6cb9bf29d311"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.389101 12024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:47:52.389838 12047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.390035 12024 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:52.390144 12024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "10fb29450da54f3f8cfff7473e333b64"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "2e84ae34a138a1ce2ae40e3f916e61dd"
server_key_iv: "c01bc4ede66f0cb6ea4a6cb9bf29d311"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.390226 12024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:52.398285 12024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:52.400347 12024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:52.400497 12024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:52.400696 12024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:52.400986 12024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:52.401019 12024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.401039 12024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:52.401086 12024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.406484 12024 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:33079
I20260812 03:47:52.406553 12251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:33079 every 8 connection(s)
I20260812 03:47:52.406831 12024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:47:52.409623 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 12024
I20260812 03:47:52.409729 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:47:52.409935 7698 external_mini_cluster.cc:1428] Setting key 04ae841e8b128be400ce2415bb444bf7
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:52.415935 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:52.407466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:50901 (local address 127.7.132.190:46109)
0812 03:47:52.407557 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:52.407561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:52.407700 (+ 139us) server_negotiation.cc:407] Connection header received
0812 03:47:52.408146 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:52.408148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:52.408160 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:52.408221 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:52.409075 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.409648 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.411011 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.411316 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.411387 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:52.413375 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:52.413376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:52.413377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:52.413384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:52.414618 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:52.415007 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:52.415008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:52.415009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:52.415065 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:52.415299 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:52.415299 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:52.415300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:52.415429 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:52.415496 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:52.415765 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 03:47:52.415851 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:47:52.416069 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:52.407416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46109 (local address 127.7.132.130:50901)
0812 03:47:52.407575 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:52.407613 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:52.408048 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:52.408282 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:52.408285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:52.408810 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:52.408973 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:52.408977 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.409750 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.409751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:52.410927 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:52.410929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.411404 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.411405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:52.411493 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:52.411934 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:52.411942 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:52.413287 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:52.414728 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:52.414734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:52.414736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:52.414934 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:52.415130 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:52.415131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:52.415131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:52.415235 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:52.415500 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:52.415506 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:52.415630 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 03:47:52.415724 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:47:52.415867 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260812 03:47:52.416493 12252 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46109
I20260812 03:47:52.416601 12252 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:52.416783 12252 heartbeater.cc:511] Master 127.7.132.190:46109 requested a full tablet report, sending...
I20260812 03:47:52.417122 11706 ts_manager.cc:194] Registered new tserver with Master: 10fb29450da54f3f8cfff7473e333b64 (127.7.132.130:33079)
I20260812 03:47:52.417472 11706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:50901
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:47:52.435009 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46109
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:52.514135 12262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:52.514300 12262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:52.514321 12262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:52.515456 12262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:52.515499 12262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:52.515524 12262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:47:52.516788 12262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:52.516942 12262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:52.517091 12262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:52.521569 12267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:52.521606 12270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:52.521669 12262 server_base.cc:1034] running on GCE node
W20260812 03:47:52.521628 12272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:52.521948 12262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:52.522164 12262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:52.523295 12262 hybrid_clock.cc:648] HybridClock initialized: now 1786506472523272 us; error 40 us; skew 500 ppm
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[11631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:52.525622 12262 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:47:52.526093 12262 webserver.cc:460] Webserver started at http://127.7.132.131:37417/ using document root <none> and password file <none>
I20260812 03:47:52.526310 12262 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:52.526353 12262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:52.526494 12262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:52.527580 12262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "5a2d957f05ad407891271d34f8cbe87d"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "a08f6af6a1f241f1b1dbe4b7615e1e67"
server_key_iv: "7bd4b5941c41119d99dd9a4019456b0d"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.527940 12262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "5a2d957f05ad407891271d34f8cbe87d"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "a08f6af6a1f241f1b1dbe4b7615e1e67"
server_key_iv: "7bd4b5941c41119d99dd9a4019456b0d"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.529203 12262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:47:52.529915 12282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.530097 12262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:52.530205 12262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "5a2d957f05ad407891271d34f8cbe87d"
format_stamp: "Formatted at 2026-08-12 03:47:52 on dist-test-slave-39l8"
server_key: "a08f6af6a1f241f1b1dbe4b7615e1e67"
server_key_iv: "7bd4b5941c41119d99dd9a4019456b0d"
server_key_version: "encryptionkey@0"
I20260812 03:47:52.530292 12262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:52.540033 12262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:52.541673 12262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:52.541824 12262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:52.542033 12262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:52.542317 12262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:52.542371 12262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.542410 12262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:52.542438 12262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:52.548250 12262 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:41283
I20260812 03:47:52.548297 12482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:41283 every 8 connection(s)
I20260812 03:47:52.548590 12262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:47:52.549567 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 12262
I20260812 03:47:52.549671 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:47:52.549850 7698 external_mini_cluster.cc:1428] Setting key 8aa540dc8bd86bdb9bf1ce9d4b74344d
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[11631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506472, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:52.556811 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:52.549228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:46349 (local address 127.7.132.190:46109)
0812 03:47:52.549295 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:52.549297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:52.549513 (+ 216us) server_negotiation.cc:407] Connection header received
0812 03:47:52.550000 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:52.550001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:52.550014 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:52.550101 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:52.551197 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.551680 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.552561 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.552738 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.552786 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:52.554644 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:52.554646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:52.554646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:52.554653 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:52.555841 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:52.556153 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:52.556153 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:52.556154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:52.556197 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:52.556378 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:52.556379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:52.556379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:52.556465 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:52.556510 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:52.556701 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 03:47:52.556755 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:47:52.556967 12485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:52.549215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46109 (local address 127.7.132.131:46349)
0812 03:47:52.549376 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:52.549428 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:52.549875 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:52.550170 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:52.550175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:52.550930 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:52.551106 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:52.551110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.551758 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.551759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:52.552485 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:52.552487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:52.552806 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:52.552807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:52.552894 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:52.553271 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:52.553279 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:52.554543 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:52.555906 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:52.555911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:52.555912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:52.556093 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:52.556249 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:52.556249 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:52.556250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:52.556330 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:52.556512 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:52.556516 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:52.556603 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 03:47:52.556673 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:47:52.556776 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260812 03:47:52.557421 12483 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46109
I20260812 03:47:52.557507 12483 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:52.557654 12483 heartbeater.cc:511] Master 127.7.132.190:46109 requested a full tablet report, sending...
I20260812 03:47:52.558058 11706 ts_manager.cc:194] Registered new tserver with Master: 5a2d957f05ad407891271d34f8cbe87d (127.7.132.131:41283)
I20260812 03:47:52.558418 11706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:46349
I20260812 03:47:52.561391 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:47:52.573508 11795 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:47:52.572617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.7.132.190:46109)
0812 03:47:52.572695 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:52.572698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:52.572788 (+ 90us) server_negotiation.cc:407] Connection header received
0812 03:47:52.572879 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:52.572880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:52.572890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:52.572957 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:52.573456 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34428: BlockingRecv error: recv got EOF from 127.0.0.1:34428 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:47:52.574302 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 11800
I20260812 03:47:52.594305 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 12024
I20260812 03:47:52.601982 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 12262
I20260812 03:47:52.622924 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 11653
2026-08-12T03:47:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1984 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[12500](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[12500](info): set up 2 sockets
Aug 12 03:47:52 dist-test-slave-39l8 krb5kdc[12500](info): commencing operation
krb5kdc: starting...
W20260812 03:47:53.824947 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:47:53 dist-test-slave-39l8 krb5kdc[12500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:47:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:47:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:47:53.932799 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43827
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:54.011349 12526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:54.011576 12526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:54.011612 12526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:54.013640 12526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:47:54.013687 12526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:54.013706 12526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:54.013727 12526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:47:54.013753 12526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:47:54.016108 12526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:54.016412 12526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:54.016664 12526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:54.022418 12534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:54.022439 12536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:54.022584 12526 server_base.cc:1034] running on GCE node
W20260812 03:47:54.022436 12533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:54.022867 12526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:54.023169 12526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:54.024338 12526 hybrid_clock.cc:648] HybridClock initialized: now 1786506474024320 us; error 31 us; skew 500 ppm
Aug 12 03:47:54 dist-test-slave-39l8 krb5kdc[12500](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506474, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:54.026813 12526 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:47:54.027212 12526 webserver.cc:460] Webserver started at http://127.7.132.190:36373/ using document root <none> and password file <none>
I20260812 03:47:54.027376 12526 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:54.027413 12526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:54.027493 12526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:54.028426 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "bedec3845960444d91b33a03b0b3404b"
format_stamp: "Formatted at 2026-08-12 03:47:54 on dist-test-slave-39l8"
server_key: "22ae6926ed7fa36a73e32bcb15b198fe"
server_key_iv: "062b878e8c418984a486bc6b1c888873"
server_key_version: "encryptionkey@0"
I20260812 03:47:54.028707 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "bedec3845960444d91b33a03b0b3404b"
format_stamp: "Formatted at 2026-08-12 03:47:54 on dist-test-slave-39l8"
server_key: "22ae6926ed7fa36a73e32bcb15b198fe"
server_key_iv: "062b878e8c418984a486bc6b1c888873"
server_key_version: "encryptionkey@0"
I20260812 03:47:54.029841 12526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:47:54.030500 12546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:54.030666 12526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:54.030776 12526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "bedec3845960444d91b33a03b0b3404b"
format_stamp: "Formatted at 2026-08-12 03:47:54 on dist-test-slave-39l8"
server_key: "22ae6926ed7fa36a73e32bcb15b198fe"
server_key_iv: "062b878e8c418984a486bc6b1c888873"
server_key_version: "encryptionkey@0"
I20260812 03:47:54.030843 12526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:54.038374 12526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:54.040067 12526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:54.040177 12526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:54.043861 12526 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:43827
I20260812 03:47:54.043952 12640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:43827 every 8 connection(s)
I20260812 03:47:54.044229 12526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:47:54.045074 12642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:54.047130 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 12526
I20260812 03:47:54.047209 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:47:54.047350 12642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b: Bootstrap starting.
I20260812 03:47:54.047369 7698 external_mini_cluster.cc:1428] Setting key 0884430cc755894059c901e13f9bb2d4
I20260812 03:47:54.048033 12642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:54.048314 12642 log.cc:824] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:54.049067 12642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b: No bootstrap required, opened a new log
I20260812 03:47:54.051159 12642 raft_consensus.cc:348] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedec3845960444d91b33a03b0b3404b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43827 } }
I20260812 03:47:54.051334 12642 raft_consensus.cc:374] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:54.051378 12642 raft_consensus.cc:729] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bedec3845960444d91b33a03b0b3404b, State: Initialized, Role: FOLLOWER
I20260812 03:47:54.051546 12642 consensus_queue.cc:260] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedec3845960444d91b33a03b0b3404b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43827 } }
I20260812 03:47:54.051640 12642 raft_consensus.cc:388] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:47:54.051685 12642 raft_consensus.cc:482] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:47:54.051738 12642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:54.052462 12642 raft_consensus.cc:504] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedec3845960444d91b33a03b0b3404b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43827 } }
I20260812 03:47:54.052583 12642 leader_election.cc:302] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bedec3845960444d91b33a03b0b3404b; no voters:
I20260812 03:47:54.052800 12642 leader_election.cc:288] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:47:54.052868 12649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:54.053063 12649 raft_consensus.cc:686] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [term 1 LEADER]: Becoming Leader. State: Replica: bedec3845960444d91b33a03b0b3404b, State: Running, Role: LEADER
I20260812 03:47:54.053088 12642 sys_catalog.cc:564] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:47:54.053169 12649 consensus_queue.cc:237] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedec3845960444d91b33a03b0b3404b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43827 } }
I20260812 03:47:54.053496 12649 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [sys.catalog]: SysCatalogTable state changed. Reason: New leader bedec3845960444d91b33a03b0b3404b. Latest consensus state: current_term: 1 leader_uuid: "bedec3845960444d91b33a03b0b3404b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedec3845960444d91b33a03b0b3404b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43827 } } }
I20260812 03:47:54.053505 12651 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bedec3845960444d91b33a03b0b3404b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedec3845960444d91b33a03b0b3404b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43827 } } }
I20260812 03:47:54.053587 12649 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:54.053606 12651 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:54.053817 12658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:47:54.054324 12658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:47:54 dist-test-slave-39l8 krb5kdc[12500](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:54.058404 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:54.047841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.7.132.190:43827)
0812 03:47:54.047993 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:54.048000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:54.048022 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:47:54.048470 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:54.048477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:54.048547 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:54.048694 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:54.049423 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:54.050224 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:54.051395 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:54.051901 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:54.051979 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:54.055329 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:54.055333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:54.055333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:54.055343 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:54.056968 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:54.057353 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:54.057355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:54.057355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:54.057431 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:54.057626 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:54.057626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:54.057626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:54.057823 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:54.057913 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:54.058057 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:47:54.058147 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260812 03:47:54.059332 12658 catalog_manager.cc:1347] Generated new cluster ID: 17e5931684b9412c8eeb707b43326fff
I20260812 03:47:54.059379 12658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:47:54.059442 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:47:54.068020 12658 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:47:54.068492 12658 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:47:54.078586 12647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:54.074270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.7.132.190:43827)
0812 03:47:54.074323 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:54.074325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:54.074488 (+ 163us) server_negotiation.cc:407] Connection header received
0812 03:47:54.074550 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:54.074551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:54.074561 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:54.074628 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:54.075055 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:54.075817 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:54.076840 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:54.077140 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:54.077200 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:54.077947 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:54.077948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:54.077949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:47:54.078156 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:54.078176 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:47:54.078183 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:54.078245 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:54.078480 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 03:47:54.078507 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:47:54.078580 12658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bedec3845960444d91b33a03b0b3404b: Generated new TSK 0
I20260812 03:47:54.078817 12658 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:47:54.082435 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 12526
2026-08-12T03:47:54Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1464 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:54 dist-test-slave-39l8 krb5kdc[12699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:54 dist-test-slave-39l8 krb5kdc[12699](info): set up 2 sockets
Aug 12 03:47:54 dist-test-slave-39l8 krb5kdc[12699](info): commencing operation
krb5kdc: starting...
W20260812 03:47:55.283780 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:47:55 dist-test-slave-39l8 krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:47:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:47:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:47:55.394480 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33507
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:55.472118 12723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:55.472347 12723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:55.472380 12723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:55.473455 12723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:47:55.473520 12723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:55.473544 12723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:55.473565 12723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:47:55.473594 12723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:47:55.475039 12723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:55.475318 12723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:55.475559 12723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:55.480396 12732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:55.480440 12723 server_base.cc:1034] running on GCE node
W20260812 03:47:55.480403 12736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:55.480396 12731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:55.480796 12723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:55.481112 12723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:55.482281 12723 hybrid_clock.cc:648] HybridClock initialized: now 1786506475482271 us; error 39 us; skew 500 ppm
Aug 12 03:47:55 dist-test-slave-39l8 krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:55.484692 12723 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:47:55.485116 12723 webserver.cc:460] Webserver started at http://127.7.132.190:37129/ using document root <none> and password file <none>
I20260812 03:47:55.485335 12723 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:55.485401 12723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:55.485549 12723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:55.486536 12723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "0d9d14798bf64c1fbc14f857feaa0d40"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "cf69195ab012f296b766f633cd588c59"
server_key_iv: "d7cfcc0d7273a7bf81138bb2a4cc9da7"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.486925 12723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "0d9d14798bf64c1fbc14f857feaa0d40"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "cf69195ab012f296b766f633cd588c59"
server_key_iv: "d7cfcc0d7273a7bf81138bb2a4cc9da7"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.488198 12723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:47:55.488919 12748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:55.489091 12723 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:55.489210 12723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "0d9d14798bf64c1fbc14f857feaa0d40"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "cf69195ab012f296b766f633cd588c59"
server_key_iv: "d7cfcc0d7273a7bf81138bb2a4cc9da7"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.489283 12723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:55.510630 12723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:55.512504 12723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:55.512683 12723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:55.516474 12723 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:33507
I20260812 03:47:55.516528 12826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:33507 every 8 connection(s)
I20260812 03:47:55.516867 12723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:47:55.517952 12827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:55.518940 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 12723
I20260812 03:47:55.519037 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:47:55.519227 7698 external_mini_cluster.cc:1428] Setting key e54333709a38d8bc9d4cdc19e772a673
I20260812 03:47:55.520855 12827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40: Bootstrap starting.
I20260812 03:47:55.521543 12827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:55.521867 12827 log.cc:824] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:55.522588 12827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40: No bootstrap required, opened a new log
I20260812 03:47:55.524840 12827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d9d14798bf64c1fbc14f857feaa0d40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33507 } }
I20260812 03:47:55.524988 12827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:55.525028 12827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9d14798bf64c1fbc14f857feaa0d40, State: Initialized, Role: FOLLOWER
I20260812 03:47:55.525221 12827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d9d14798bf64c1fbc14f857feaa0d40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33507 } }
I20260812 03:47:55.525306 12827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:47:55.525349 12827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:47:55.525403 12827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:55.526104 12827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d9d14798bf64c1fbc14f857feaa0d40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33507 } }
I20260812 03:47:55.526227 12827 leader_election.cc:302] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d9d14798bf64c1fbc14f857feaa0d40; no voters:
Aug 12 03:47:55 dist-test-slave-39l8 krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:55.526384 12827 leader_election.cc:288] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:47:55.526448 12840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:55.526656 12840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [term 1 LEADER]: Becoming Leader. State: Replica: 0d9d14798bf64c1fbc14f857feaa0d40, State: Running, Role: LEADER
I20260812 03:47:55.526655 12827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:47:55.526787 12840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d9d14798bf64c1fbc14f857feaa0d40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33507 } }
I20260812 03:47:55.527123 12841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d9d14798bf64c1fbc14f857feaa0d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d9d14798bf64c1fbc14f857feaa0d40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33507 } } }
I20260812 03:47:55.527220 12841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:55.527470 12843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d9d14798bf64c1fbc14f857feaa0d40. Latest consensus state: current_term: 1 leader_uuid: "0d9d14798bf64c1fbc14f857feaa0d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d9d14798bf64c1fbc14f857feaa0d40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33507 } } }
I20260812 03:47:55.527812 12843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:55.528002 12852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:47:55.528524 12852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:47:55.530526 12852 catalog_manager.cc:1347] Generated new cluster ID: ba64aa3cb3244705b38229daea369ae5
I20260812 03:47:55.530575 12852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:47:55.531179 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:55.519753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34586 (local address 127.7.132.190:33507)
0812 03:47:55.519991 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:55.520001 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:55.520033 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:47:55.520560 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:55.520569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:55.520631 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:55.520787 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:55.521421 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.522758 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.523470 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.524089 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.524215 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:55.527007 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:55.527011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:55.527012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:55.527023 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:55.529217 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:55.529648 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:55.529651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:55.529651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:55.529753 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:55.529988 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:55.529989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:55.529990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:55.530299 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:55.530466 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:55.530842 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 03:47:55.530970 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":113,"threads_started":1}
I20260812 03:47:55.551321 12852 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:47:55.552079 12852 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:47:55.557065 12852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d9d14798bf64c1fbc14f857feaa0d40: Generated new TSK 0
I20260812 03:47:55.557264 12852 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:47:55.585235 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33507
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:55.662724 12874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:55.662920 12874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:55.662941 12874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:55.663985 12874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:55.664023 12874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:55.664077 12874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:47:55.665393 12874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:55.665596 12874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:55.665812 12874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:55.670409 12887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:55.670382 12890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:55.670416 12874 server_base.cc:1034] running on GCE node
W20260812 03:47:55.670387 12882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:55.670786 12874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:55.671031 12874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:55.672179 12874 hybrid_clock.cc:648] HybridClock initialized: now 1786506475672166 us; error 34 us; skew 500 ppm
Aug 12 03:47:55 dist-test-slave-39l8 krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:55.674532 12874 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:47:55.675009 12874 webserver.cc:460] Webserver started at http://127.7.132.129:39131/ using document root <none> and password file <none>
I20260812 03:47:55.675217 12874 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:55.675261 12874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:55.675396 12874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:55.676425 12874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "1c042bed96834378b2f9f3374e4fe7d9"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "d44715cd7bb72f16251ec5ecba670c76"
server_key_iv: "4407a88b64cbbd35c3c2ec5e20f9e5e0"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.676812 12874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "1c042bed96834378b2f9f3374e4fe7d9"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "d44715cd7bb72f16251ec5ecba670c76"
server_key_iv: "4407a88b64cbbd35c3c2ec5e20f9e5e0"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.678117 12874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:47:55.678818 12900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:55.678985 12874 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:55.679095 12874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "1c042bed96834378b2f9f3374e4fe7d9"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "d44715cd7bb72f16251ec5ecba670c76"
server_key_iv: "4407a88b64cbbd35c3c2ec5e20f9e5e0"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.679178 12874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:55.694007 12874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:55.695773 12874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:55.695921 12874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:55.696123 12874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:55.696410 12874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:55.696460 12874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:55.696488 12874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:55.696516 12874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:55.702284 12874 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:46837
I20260812 03:47:55.702345 13085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:46837 every 8 connection(s)
I20260812 03:47:55.702695 12874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:47:55 dist-test-slave-39l8 krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:55.709746 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 12874
I20260812 03:47:55.709844 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:47:55.710079 7698 external_mini_cluster.cc:1428] Setting key fe6d3fe7519d053c0f34efc6904d265c
I20260812 03:47:55.710974 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:55.703455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:34283 (local address 127.7.132.190:33507)
0812 03:47:55.703516 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:55.703518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:55.703677 (+ 159us) server_negotiation.cc:407] Connection header received
0812 03:47:55.704100 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:55.704101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:55.704110 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:55.704167 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:55.704994 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.705444 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.706275 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.706445 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.706488 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:55.708343 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:55.708344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:55.708344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:55.708349 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:55.709519 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:55.709893 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:55.709895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:55.709895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:55.709959 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:55.710276 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:55.710278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:55.710278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:55.710433 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:55.710493 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:55.710797 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 03:47:55.710884 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:47:55.711148 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:55.703408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33507 (local address 127.7.132.129:34283)
0812 03:47:55.703563 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:55.703601 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:55.704025 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:55.704213 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:55.704216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:55.704751 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:55.704926 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:55.704930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.705515 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.705516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:55.706202 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:55.706204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.706506 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.706507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:55.706587 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:55.707012 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:55.707020 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:55.708268 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:55.709583 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:55.709589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:55.709590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:55.709798 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:55.710037 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:55.710038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:55.710038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:55.710165 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:55.710497 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:55.710504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:55.710652 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 03:47:55.710776 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 03:47:55.710927 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260812 03:47:55.711584 13088 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33507
I20260812 03:47:55.711665 13088 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:55.711830 13088 heartbeater.cc:511] Master 127.7.132.190:33507 requested a full tablet report, sending...
I20260812 03:47:55.712347 12773 ts_manager.cc:194] Registered new tserver with Master: 1c042bed96834378b2f9f3374e4fe7d9 (127.7.132.129:46837)
I20260812 03:47:55.713327 12773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:34283
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:47:55.733042 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33507
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:55.809297 13096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:55.809484 13096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:55.809505 13096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:55.810640 13096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:55.810683 13096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:55.810760 13096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:47:55.812173 13096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:55.812398 13096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:55.812633 13096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:55.817363 13103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:55.817408 13102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:55.817422 13111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:55.817489 13096 server_base.cc:1034] running on GCE node
I20260812 03:47:55.818922 13096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:55.819169 13096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:55.820331 13096 hybrid_clock.cc:648] HybridClock initialized: now 1786506475820329 us; error 38 us; skew 500 ppm
Aug 12 03:47:55 dist-test-slave-39l8 krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:55.822664 13096 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:47:55.823086 13096 webserver.cc:460] Webserver started at http://127.7.132.130:34837/ using document root <none> and password file <none>
I20260812 03:47:55.823307 13096 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:55.823382 13096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:55.823521 13096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:55.824501 13096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "7837643aa1744e4e90875df2754286a9"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "76cc5682565e9e4f00df19b6fd8e30ee"
server_key_iv: "0ab1372c3f81060ce2d148dd1df50d53"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.824872 13096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "7837643aa1744e4e90875df2754286a9"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "76cc5682565e9e4f00df19b6fd8e30ee"
server_key_iv: "0ab1372c3f81060ce2d148dd1df50d53"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.826136 13096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 03:47:55.827044 13122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:55.827224 13096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:55.827323 13096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "7837643aa1744e4e90875df2754286a9"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "76cc5682565e9e4f00df19b6fd8e30ee"
server_key_iv: "0ab1372c3f81060ce2d148dd1df50d53"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.827404 13096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:55.847823 13096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:55.849395 13096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:55.849543 13096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:55.849771 13096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:55.850060 13096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:55.850109 13096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:55.850144 13096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:55.850211 13096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:55.855800 13096 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:41605
I20260812 03:47:55.855857 13330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:41605 every 8 connection(s)
I20260812 03:47:55.856114 13096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:47:55.857532 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 13096
I20260812 03:47:55.857635 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:47:55.857838 7698 external_mini_cluster.cc:1428] Setting key 5ce67ca87c74b4652af5339cd7a41ac4
Aug 12 03:47:55 dist-test-slave-39l8 krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:55.865465 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:55.856715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:34025 (local address 127.7.132.190:33507)
0812 03:47:55.856797 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:55.856799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:55.857017 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:47:55.857488 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:55.857489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:55.857500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:55.857555 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:55.858687 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.859201 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.860169 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.860393 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.860480 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:55.862737 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:55.862739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:55.862739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:55.862746 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:55.864093 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:55.864509 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:55.864510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:55.864511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:55.864571 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:55.864825 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:55.864826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:55.864826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:55.864976 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:55.865040 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:55.865307 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 03:47:55.865384 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:47:55.865782 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:55.856715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33507 (local address 127.7.132.130:34025)
0812 03:47:55.856873 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:55.856930 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:55.857375 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:55.857615 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:55.857619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:55.858347 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:55.858599 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:55.858604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.859317 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.859318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:55.860056 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:55.860059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:55.860492 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:55.860493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:55.860623 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:55.861087 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:55.861095 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:55.862605 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:55.864194 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:55.864200 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:55.864202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:55.864419 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:55.864640 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:55.864640 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:55.864641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:55.864751 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:55.865044 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:55.865050 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:55.865178 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 03:47:55.865273 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:47:55.865411 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260812 03:47:55.866231 13332 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33507
I20260812 03:47:55.866313 13332 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:55.866487 13332 heartbeater.cc:511] Master 127.7.132.190:33507 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
I20260812 03:47:55.866859 12773 ts_manager.cc:194] Registered new tserver with Master: 7837643aa1744e4e90875df2754286a9 (127.7.132.130:41605)
I20260812 03:47:55.867249 12773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:34025
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:47:55.881959 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33507
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:55.960490 13337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:55.960695 13337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:55.960717 13337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:55.961836 13337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:55.961877 13337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:55.961930 13337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:47:55.963431 13337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:55.963681 13337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:55.963945 13337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:55.968886 13349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:55.968880 13345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:55.968878 13347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:55.968905 13337 server_base.cc:1034] running on GCE node
I20260812 03:47:55.969251 13337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:55.969498 13337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:55.970680 13337 hybrid_clock.cc:648] HybridClock initialized: now 1786506475970656 us; error 42 us; skew 500 ppm
Aug 12 03:47:55 dist-test-slave-39l8 krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:55.973141 13337 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:47:55.973627 13337 webserver.cc:460] Webserver started at http://127.7.132.131:33431/ using document root <none> and password file <none>
I20260812 03:47:55.973855 13337 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:55.973896 13337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:55.974056 13337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:55.975109 13337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "0ff07487525d456b955090eb0f06a259"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "02ba08b9eababeacdf4151e153fa3b41"
server_key_iv: "1bb0f1cc923ae775907be25a17fd8061"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.975459 13337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "0ff07487525d456b955090eb0f06a259"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "02ba08b9eababeacdf4151e153fa3b41"
server_key_iv: "1bb0f1cc923ae775907be25a17fd8061"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.976717 13337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:47:55.977449 13365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:55.977644 13337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:55.977737 13337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "0ff07487525d456b955090eb0f06a259"
format_stamp: "Formatted at 2026-08-12 03:47:55 on dist-test-slave-39l8"
server_key: "02ba08b9eababeacdf4151e153fa3b41"
server_key_iv: "1bb0f1cc923ae775907be25a17fd8061"
server_key_version: "encryptionkey@0"
I20260812 03:47:55.977790 13337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:56.006675 13337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:56.009130 13337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:56.009245 13337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:56.009425 13337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:56.009765 13337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:56.009802 13337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:56.009848 13337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:56.009876 13337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:56.017750 13337 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:35037
I20260812 03:47:56.017815 13559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:35037 every 8 connection(s)
I20260812 03:47:56.018285 13337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:47:56 dist-test-slave-39l8 krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506475, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:56.026670 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 13337
I20260812 03:47:56.026820 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:47:56.027040 7698 external_mini_cluster.cc:1428] Setting key 28902293c0909486f56b7bcb79d0116b
I20260812 03:47:56.029049 13565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:56.019019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33507 (local address 127.7.132.131:51819)
0812 03:47:56.019332 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:56.019384 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:56.020056 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:56.020450 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:56.020455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:56.021261 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:56.021600 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:56.021607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:56.022552 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:56.022553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:56.023408 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:56.023410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:56.023828 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:56.023830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:56.023943 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:56.024366 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:56.024374 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:56.025683 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:56.027559 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:56.027566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:56.027568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:56.027900 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:56.028208 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:56.028209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:56.028210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:56.028363 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:56.028595 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:56.028597 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:56.028702 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 03:47:56.028778 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:47:56.028895 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":100,"threads_started":1}
I20260812 03:47:56.029210 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:56.019063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:51819 (local address 127.7.132.190:33507)
0812 03:47:56.019180 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:56.019182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:56.019475 (+ 293us) server_negotiation.cc:407] Connection header received
0812 03:47:56.020264 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:56.020265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:56.020281 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:56.020346 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:56.021706 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:56.022455 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:56.023469 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:56.023663 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:56.023725 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:56.025769 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:56.025770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:56.025770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:56.025777 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:56.027477 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:56.028074 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:56.028074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:56.028075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:56.028124 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:56.028434 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:56.028435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:56.028435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:56.028533 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:56.028603 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:56.028956 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 03:47:56.029040 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:47:56.029660 13560 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33507
I20260812 03:47:56.029779 13560 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:56.029955 13560 heartbeater.cc:511] Master 127.7.132.190:33507 requested a full tablet report, sending...
I20260812 03:47:56.030297 12773 ts_manager.cc:194] Registered new tserver with Master: 0ff07487525d456b955090eb0f06a259 (127.7.132.131:35037)
I20260812 03:47:56.030623 12773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:51819
I20260812 03:47:56.038817 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:47:56 dist-test-slave-39l8 krb5kdc[12699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506476, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 03:47:56 dist-test-slave-39l8 krb5kdc[12699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506476, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:56.062528 12835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:56.055592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34592 (local address 127.7.132.190:33507)
0812 03:47:56.055673 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:56.055675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:56.055775 (+ 100us) server_negotiation.cc:407] Connection header received
0812 03:47:56.055812 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:56.055813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:56.055821 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:56.055876 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:56.056420 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:56.056992 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:56.057839 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:56.058032 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:56.058109 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:56.060244 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:56.060246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:56.060246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:56.060253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:56.061601 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:56.061889 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:56.061890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:56.061890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:56.061938 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:56.062084 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:56.062084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:56.062084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:56.062209 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:56.062285 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:56.062369 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:47:56.062428 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 03:47:56.062822 12773 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34592
I20260812 03:47:56.063288 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 12874
I20260812 03:47:56.081451 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 13096
I20260812 03:47:56.088294 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 13337
I20260812 03:47:56.094115 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 12723
2026-08-12T03:47:56Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2010 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:56 dist-test-slave-39l8 krb5kdc[13582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:56 dist-test-slave-39l8 krb5kdc[13582](info): set up 2 sockets
Aug 12 03:47:56 dist-test-slave-39l8 krb5kdc[13582](info): commencing operation
krb5kdc: starting...
W20260812 03:47:57.397131 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 03:47:57.543744 7698 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.146s user 0.000s sys 0.003s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 03:47:57.809641 7698 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.266s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 03:47:57.925489 7698 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.116s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/test-user.keytab.
W20260812 03:47:58.223869 7698 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.298s user 0.000s sys 0.002s
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:47:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:47:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:47:58.267191 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43813
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:58.344826 13605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:58.345045 13605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:58.345086 13605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:58.346274 13605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:47:58.346340 13605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:58.346365 13605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:58.346393 13605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:47:58.346424 13605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:47:58.347885 13605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:58.348145 13605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:58.348356 13605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:58.353117 13617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:58.353173 13616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:58.353186 13605 server_base.cc:1034] running on GCE node
W20260812 03:47:58.353123 13622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:58.353572 13605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:58.353803 13605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:58.354950 13605 hybrid_clock.cc:648] HybridClock initialized: now 1786506478354922 us; error 28 us; skew 500 ppm
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:58.361642 13605 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:47:58.362075 13605 webserver.cc:460] Webserver started at http://127.7.132.190:40079/ using document root <none> and password file <none>
I20260812 03:47:58.362298 13605 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:58.362362 13605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:58.362488 13605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:58.363520 13605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "0cf7ffd19c5d45849e0e606ae9c48c40"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "72b304a3a72b23a6c9ad358312b81462"
server_key_iv: "607ea399efe480d7466ad6d81de56594"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.363905 13605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "0cf7ffd19c5d45849e0e606ae9c48c40"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "72b304a3a72b23a6c9ad358312b81462"
server_key_iv: "607ea399efe480d7466ad6d81de56594"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.365190 13605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:47:58.365903 13632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.366076 13605 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:47:58.366180 13605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "0cf7ffd19c5d45849e0e606ae9c48c40"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "72b304a3a72b23a6c9ad358312b81462"
server_key_iv: "607ea399efe480d7466ad6d81de56594"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.366251 13605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:58.375262 13605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:58.377434 13605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:58.377586 13605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:58.381592 13605 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:43813
I20260812 03:47:58.381636 13720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:43813 every 8 connection(s)
I20260812 03:47:58.382190 13605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:47:58.382911 13728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:47:58.385035 13728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40: Bootstrap starting.
I20260812 03:47:58.385547 13728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40: Neither blocks nor log segments found. Creating new log.
I20260812 03:47:58.385901 13728 log.cc:824] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40: Log is configured to *not* fsync() on all Append() calls
I20260812 03:47:58.386572 13728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40: No bootstrap required, opened a new log
I20260812 03:47:58.388116 13728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf7ffd19c5d45849e0e606ae9c48c40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43813 } }
I20260812 03:47:58.388244 13728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:47:58.388266 13728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf7ffd19c5d45849e0e606ae9c48c40, State: Initialized, Role: FOLLOWER
I20260812 03:47:58.388358 13728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf7ffd19c5d45849e0e606ae9c48c40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43813 } }
I20260812 03:47:58.388407 13728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:47:58.388427 13728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:47:58.388451 13728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:47:58.389320 13728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf7ffd19c5d45849e0e606ae9c48c40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43813 } }
I20260812 03:47:58.389410 13728 leader_election.cc:302] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cf7ffd19c5d45849e0e606ae9c48c40; no voters:
I20260812 03:47:58.389531 13728 leader_election.cc:288] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:47:58.389592 13733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:47:58.389734 13733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [term 1 LEADER]: Becoming Leader. State: Replica: 0cf7ffd19c5d45849e0e606ae9c48c40, State: Running, Role: LEADER
I20260812 03:47:58.389851 13733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf7ffd19c5d45849e0e606ae9c48c40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43813 } }
I20260812 03:47:58.389855 13728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:47:58.390208 13735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cf7ffd19c5d45849e0e606ae9c48c40. Latest consensus state: current_term: 1 leader_uuid: "0cf7ffd19c5d45849e0e606ae9c48c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf7ffd19c5d45849e0e606ae9c48c40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43813 } } }
I20260812 03:47:58.390301 13735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:58.390205 13734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cf7ffd19c5d45849e0e606ae9c48c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf7ffd19c5d45849e0e606ae9c48c40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43813 } } }
I20260812 03:47:58.390466 13734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:47:58.390910 13739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:47:58.391386 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 13605
I20260812 03:47:58.391450 13739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:47:58.391490 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:47:58.391677 7698 external_mini_cluster.cc:1428] Setting key 58992e898d01098ce3871fa938923e48
I20260812 03:47:58.393061 13739 catalog_manager.cc:1347] Generated new cluster ID: d7cdde1107a641ce96a583ca195e9b6c
I20260812 03:47:58.393114 13739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:58.403288 13739 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:47:58.403510 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.392141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33480 (local address 127.7.132.190:43813)
0812 03:47:58.392297 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:58.392305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:58.392466 (+ 161us) server_negotiation.cc:407] Connection header received
0812 03:47:58.392871 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:58.392879 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:58.392933 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:58.393037 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:58.393820 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.395030 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.395985 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.397082 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.397163 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.399802 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:58.399805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:58.399805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:58.399815 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:58.401763 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.402135 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.402137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.402138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.402229 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.402413 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.402414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.402414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.402661 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:58.402818 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:58.403092 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 03:47:58.403234 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260812 03:47:58.403967 13739 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:47:58.407488 13739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cf7ffd19c5d45849e0e606ae9c48c40: Generated new TSK 0
I20260812 03:47:58.407639 13739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:47:58.435006 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43813
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:58.513111 13766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:58.513305 13766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:58.513329 13766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:58.514430 13766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:58.514467 13766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:58.514492 13766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:47:58.515933 13766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:58.516155 13766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:58.516376 13766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:58.520994 13780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:58.521029 13777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:58.521075 13766 server_base.cc:1034] running on GCE node
W20260812 03:47:58.520994 13774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:58.521411 13766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:58.521703 13766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:58.522845 13766 hybrid_clock.cc:648] HybridClock initialized: now 1786506478522829 us; error 43 us; skew 500 ppm
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:58.525161 13766 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:47:58.525589 13766 webserver.cc:460] Webserver started at http://127.7.132.129:42803/ using document root <none> and password file <none>
I20260812 03:47:58.525807 13766 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:58.525883 13766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:58.526008 13766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:58.527035 13766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "d1d07f8945c34219adac178ec98d4cbd"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "c06d415d57055e504b1be44d46ff70fa"
server_key_iv: "7ea0aead8f99bb00857a75faa6f759f2"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.527380 13766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "d1d07f8945c34219adac178ec98d4cbd"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "c06d415d57055e504b1be44d46ff70fa"
server_key_iv: "7ea0aead8f99bb00857a75faa6f759f2"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.528566 13766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:47:58.529294 13788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.529474 13766 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:58.529575 13766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "d1d07f8945c34219adac178ec98d4cbd"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "c06d415d57055e504b1be44d46ff70fa"
server_key_iv: "7ea0aead8f99bb00857a75faa6f759f2"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.529660 13766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:58.543365 13766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:58.545079 13766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:58.545240 13766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:58.545452 13766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:58.545771 13766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:58.545827 13766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.545866 13766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:58.545893 13766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.551764 13766 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:37029
I20260812 03:47:58.551816 13984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:37029 every 8 connection(s)
I20260812 03:47:58.552112 13766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:58.559376 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 13766
I20260812 03:47:58.559487 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:47:58.559720 7698 external_mini_cluster.cc:1428] Setting key ea476b777d2f747a6131ce676cd55ad0
I20260812 03:47:58.560705 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.552814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:46583 (local address 127.7.132.190:43813)
0812 03:47:58.552862 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:58.552864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:58.553032 (+ 168us) server_negotiation.cc:407] Connection header received
0812 03:47:58.553868 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:58.553869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:58.553880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:58.553930 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:58.554474 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.554974 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.555812 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.555994 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.556038 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.557910 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:58.557912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:58.557912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:58.557923 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:58.559180 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.559631 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.559633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.559633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.559702 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.559975 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.559977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.559977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.560120 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:58.560189 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:58.560497 (+ 308us) server_negotiation.cc:299] Negotiation successful
0812 03:47:58.560589 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:47:58.560724 13987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.552762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43813 (local address 127.7.132.129:46583)
0812 03:47:58.552923 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:58.552962 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:58.553786 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:58.554004 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:58.554007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:58.554244 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:58.554408 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:58.554412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.555058 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.555059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:58.555747 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:58.555749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.556074 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.556075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:58.556164 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.556534 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:58.556541 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:58.557814 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:58.559279 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:58.559285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:58.559287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:58.559535 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:58.559780 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:58.559781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:58.559781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:58.559906 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:58.560189 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:58.560196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:58.560328 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 03:47:58.560458 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 03:47:58.560577 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260812 03:47:58.561522 13985 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43813
I20260812 03:47:58.561635 13985 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:58.561821 13985 heartbeater.cc:511] Master 127.7.132.190:43813 requested a full tablet report, sending...
I20260812 03:47:58.562348 13664 ts_manager.cc:194] Registered new tserver with Master: d1d07f8945c34219adac178ec98d4cbd (127.7.132.129:37029)
I20260812 03:47:58.563019 13664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:46583
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:47:58.583285 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43813
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:58.661019 13990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:58.661303 13990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:58.661345 13990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:58.662653 13990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:58.662698 13990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:58.662781 13990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:47:58.664178 13990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:58.664348 13990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:58.664501 13990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:58.668980 14002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:58.669013 14004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:58.669008 14000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:58.669166 13990 server_base.cc:1034] running on GCE node
I20260812 03:47:58.669363 13990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:58.669576 13990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:58.670758 13990 hybrid_clock.cc:648] HybridClock initialized: now 1786506478670746 us; error 48 us; skew 500 ppm
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:58.673084 13990 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:47:58.673528 13990 webserver.cc:460] Webserver started at http://127.7.132.130:42227/ using document root <none> and password file <none>
I20260812 03:47:58.673759 13990 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:58.673825 13990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:58.673980 13990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:58.675025 13990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "612cca55d8f94db99aa66015fb8baf54"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "b6950a134b6d54577c36ecf02f85aa85"
server_key_iv: "b53db5eff5943b0280681679516a8eee"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.675388 13990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "612cca55d8f94db99aa66015fb8baf54"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "b6950a134b6d54577c36ecf02f85aa85"
server_key_iv: "b53db5eff5943b0280681679516a8eee"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.676685 13990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:47:58.677395 14019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.677558 13990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:47:58.677666 13990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "612cca55d8f94db99aa66015fb8baf54"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "b6950a134b6d54577c36ecf02f85aa85"
server_key_iv: "b53db5eff5943b0280681679516a8eee"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.677740 13990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:58.706430 13990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:58.708314 13990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:58.708473 13990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:58.708700 13990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:58.709002 13990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:58.709055 13990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.709095 13990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:58.709120 13990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.714917 13990 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:41643
I20260812 03:47:58.714969 14196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:41643 every 8 connection(s)
I20260812 03:47:58.715308 13990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:47:58.717788 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 13990
I20260812 03:47:58.717908 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:47:58.718083 7698 external_mini_cluster.cc:1428] Setting key 9cbf203961477e7d561cc6da05af80af
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:58.724982 14200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.716031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43813 (local address 127.7.132.130:58579)
0812 03:47:58.716231 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:58.716266 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:58.717145 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:58.717345 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:58.717348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:58.717618 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:58.717910 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:58.717915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.718789 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.718790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:58.719500 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:58.719502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.719938 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.719939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:58.720057 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.720601 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:58.720609 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:58.722007 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:58.723790 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:58.723795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:58.723797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:58.723993 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:58.724199 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:58.724200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:58.724200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:58.724287 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:58.724558 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:58.724562 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:58.724659 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:47:58.724733 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:47:58.724836 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":129,"threads_started":1}
I20260812 03:47:58.724987 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.716054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:58579 (local address 127.7.132.190:43813)
0812 03:47:58.716142 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:58.716144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:58.716351 (+ 207us) server_negotiation.cc:407] Connection header received
0812 03:47:58.717232 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:58.717233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:58.717243 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:58.717293 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:58.718034 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.718689 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.719583 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.719854 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.719925 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.722102 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:58.722104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:58.722104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:58.722112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:58.723704 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.724066 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.724067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.724067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.724130 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.724351 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.724352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.724353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.724491 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:58.724566 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:58.724780 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 03:47:58.724865 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:47:58.725528 14197 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43813
I20260812 03:47:58.725634 14197 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:58.725811 14197 heartbeater.cc:511] Master 127.7.132.190:43813 requested a full tablet report, sending...
I20260812 03:47:58.726238 13664 ts_manager.cc:194] Registered new tserver with Master: 612cca55d8f94db99aa66015fb8baf54 (127.7.132.130:41643)
I20260812 03:47:58.726745 13664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:58579
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:47:58.743548 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43813
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:47:58.822199 14204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:47:58.822384 14204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:47:58.822405 14204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:47:58.823524 14204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:47:58.823565 14204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:47:58.823616 14204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:47:58.825012 14204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:47:58.825237 14204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:47:58.825453 14204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:47:58.830255 14225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:47:58.830240 14221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:58.830438 14204 server_base.cc:1034] running on GCE node
W20260812 03:47:58.830256 14220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:47:58.830811 14204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:47:58.831080 14204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:47:58.832245 14204 hybrid_clock.cc:648] HybridClock initialized: now 1786506478832232 us; error 31 us; skew 500 ppm
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:47:58.834821 14204 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:47:58.835243 14204 webserver.cc:460] Webserver started at http://127.7.132.131:44203/ using document root <none> and password file <none>
I20260812 03:47:58.835466 14204 fs_manager.cc:359] Metadata directory not provided
I20260812 03:47:58.835536 14204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:47:58.835678 14204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:47:58.836697 14204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "fc9d4f966bfb46b8aed22279187603d6"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "86687c4d4acee9e3055ec3fd13897e36"
server_key_iv: "8cea6c5414046ae09ccc8b13b0e733cc"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.837061 14204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "fc9d4f966bfb46b8aed22279187603d6"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "86687c4d4acee9e3055ec3fd13897e36"
server_key_iv: "8cea6c5414046ae09ccc8b13b0e733cc"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.838431 14204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:47:58.839241 14236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.839401 14204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.839512 14204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "fc9d4f966bfb46b8aed22279187603d6"
format_stamp: "Formatted at 2026-08-12 03:47:58 on dist-test-slave-39l8"
server_key: "86687c4d4acee9e3055ec3fd13897e36"
server_key_iv: "8cea6c5414046ae09ccc8b13b0e733cc"
server_key_version: "encryptionkey@0"
I20260812 03:47:58.839598 14204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:47:58.860757 14204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:47:58.863240 14204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:47:58.863421 14204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:47:58.863677 14204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:47:58.863987 14204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:47:58.864042 14204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.864085 14204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:47:58.864109 14204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:47:58.870249 14204 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:35259
I20260812 03:47:58.870294 14435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:35259 every 8 connection(s)
I20260812 03:47:58.870656 14204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:58.878218 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 14204
I20260812 03:47:58.878345 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:47:58.878539 7698 external_mini_cluster.cc:1428] Setting key ac42566760e4c3c92f74e9d739a3541c
I20260812 03:47:58.880059 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.871432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:57637 (local address 127.7.132.190:43813)
0812 03:47:58.871502 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:58.871504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:58.871713 (+ 209us) server_negotiation.cc:407] Connection header received
0812 03:47:58.872723 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:58.872724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:58.872738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:58.872783 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:58.873412 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.873878 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.874847 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.875043 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.875119 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.877159 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:58.877161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:58.877162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:58.877169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:58.878689 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.879110 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.879112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.879112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.879171 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.879425 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.879427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.879427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.879546 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:58.879617 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:58.879896 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 03:47:58.879974 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:47:58.880311 14440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.871400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43813 (local address 127.7.132.131:57637)
0812 03:47:58.871583 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:47:58.871627 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:47:58.872599 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:47:58.872863 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:47:58.872867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:47:58.873127 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:47:58.873319 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:58.873323 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.873973 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.873974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:58.874759 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:47:58.874762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.875131 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.875132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:47:58.875263 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.875728 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:47:58.875736 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:47:58.877063 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:47:58.878784 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:58.878791 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:58.878793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:58.879015 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:58.879243 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:47:58.879244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:47:58.879245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:47:58.879355 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:47:58.879620 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:47:58.879627 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:47:58.879759 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 03:47:58.879864 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:47:58.880013 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260812 03:47:58.880755 14436 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43813
I20260812 03:47:58.880870 14436 heartbeater.cc:464] Registering TS with master...
I20260812 03:47:58.881045 14436 heartbeater.cc:511] Master 127.7.132.190:43813 requested a full tablet report, sending...
I20260812 03:47:58.881457 13664 ts_manager.cc:194] Registered new tserver with Master: fc9d4f966bfb46b8aed22279187603d6 (127.7.132.131:35259)
I20260812 03:47:58.881808 13664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:57637
I20260812 03:47:58.890338 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:47:58.914445 14450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.906436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.7.132.129:37029)
0812 03:47:58.906597 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:58.906602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:58.906610 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:47:58.906636 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:58.906637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:58.906652 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:58.906754 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:58.907298 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.908117 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.908878 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.909244 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.909323 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.911561 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:58.911571 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:58.911571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:58.911578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:58.913214 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.913581 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.913585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.913586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.913650 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.913863 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.913864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.913864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.914008 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:58.914189 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:58.914246 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 03:47:58.914323 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
Aug 12 03:47:58 dist-test-slave-39l8 krb5kdc[13582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:47:58.922691 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:47:58.915702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.7.132.190:43813)
0812 03:47:58.915778 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:47:58.915780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:47:58.915807 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:47:58.915878 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:47:58.915879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:47:58.915889 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:47:58.915945 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:47:58.916418 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.916912 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.917580 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:47:58.917785 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:47:58.917864 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:47:58.920108 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:47:58.920109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:47:58.920110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:47:58.920117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:47:58.921505 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.921878 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.921879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.921880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.921939 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:47:58.922143 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:47:58.922144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:47:58.922145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:47:58.922296 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:47:58.922403 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:47:58.922508 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 03:47:58.922584 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 03:47:58.922955 13664 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33496
I20260812 03:47:58.923311 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 13766
I20260812 03:47:58.940763 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 13990
I20260812 03:47:58.956676 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 14204
I20260812 03:47:58.968273 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 13605
2026-08-12T03:47:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2873 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:47:59 dist-test-slave-39l8 krb5kdc[14456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:47:59 dist-test-slave-39l8 krb5kdc[14456](info): set up 2 sockets
Aug 12 03:47:59 dist-test-slave-39l8 krb5kdc[14456](info): commencing operation
krb5kdc: starting...
W20260812 03:48:00.194864 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:48:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:48:00.309118 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46209
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:00.388978 14484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:00.389178 14484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:00.389216 14484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:00.390264 14484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:00.390297 14484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:00.390311 14484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:00.390323 14484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:00.390336 14484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:00.391806 14484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:00.392004 14484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:00.392196 14484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:00.396684 14491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:00.396711 14484 server_base.cc:1034] running on GCE node
W20260812 03:48:00.396667 14490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:00.396656 14493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:00.397042 14484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:00.397378 14484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:00.398548 14484 hybrid_clock.cc:648] HybridClock initialized: now 1786506480398534 us; error 41 us; skew 500 ppm
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:00.401000 14484 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:48:00.401377 14484 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:00.401448 14484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:00.401584 14484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:00.402626 14484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "753dc33555fc40c8bdd80487ee307bbc"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "27ce71cbbc79cc92ffb07df08b2b52af"
server_key_iv: "6aa8c3f7cc134b493f5f5a035ff80d3e"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.403055 14484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "753dc33555fc40c8bdd80487ee307bbc"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "27ce71cbbc79cc92ffb07df08b2b52af"
server_key_iv: "6aa8c3f7cc134b493f5f5a035ff80d3e"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.404342 14484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:00.405069 14502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.405253 14484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:00.405356 14484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "753dc33555fc40c8bdd80487ee307bbc"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "27ce71cbbc79cc92ffb07df08b2b52af"
server_key_iv: "6aa8c3f7cc134b493f5f5a035ff80d3e"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.405445 14484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:00.420774 14484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:00.422580 14484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:00.422775 14484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:00.426923 14484 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46209
I20260812 03:48:00.426987 14582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46209 every 8 connection(s)
I20260812 03:48:00.427342 14484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:00.428762 14583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:00.432307 14583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc: Bootstrap starting.
I20260812 03:48:00.433179 14583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:00.433605 14583 log.cc:824] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:00.433856 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 14484
I20260812 03:48:00.433974 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:00.434186 7698 external_mini_cluster.cc:1428] Setting key 0de45be19653e6b8d59a57daa1017885
I20260812 03:48:00.434537 14583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc: No bootstrap required, opened a new log
I20260812 03:48:00.437808 14583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753dc33555fc40c8bdd80487ee307bbc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46209 } }
I20260812 03:48:00.437969 14583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:00.437999 14583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753dc33555fc40c8bdd80487ee307bbc, State: Initialized, Role: FOLLOWER
I20260812 03:48:00.438112 14583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753dc33555fc40c8bdd80487ee307bbc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46209 } }
I20260812 03:48:00.438181 14583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:00.438215 14583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:00.438246 14583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:00.438961 14583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753dc33555fc40c8bdd80487ee307bbc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46209 } }
I20260812 03:48:00.439044 14583 leader_election.cc:302] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 753dc33555fc40c8bdd80487ee307bbc; no voters:
I20260812 03:48:00.439205 14583 leader_election.cc:288] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:00.439229 14591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:00.439353 14591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [term 1 LEADER]: Becoming Leader. State: Replica: 753dc33555fc40c8bdd80487ee307bbc, State: Running, Role: LEADER
I20260812 03:48:00.439431 14591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753dc33555fc40c8bdd80487ee307bbc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46209 } }
I20260812 03:48:00.439459 14583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:00.439775 14591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "753dc33555fc40c8bdd80487ee307bbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753dc33555fc40c8bdd80487ee307bbc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46209 } } }
I20260812 03:48:00.441002 14591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:00.440850 14592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 753dc33555fc40c8bdd80487ee307bbc. Latest consensus state: current_term: 1 leader_uuid: "753dc33555fc40c8bdd80487ee307bbc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "753dc33555fc40c8bdd80487ee307bbc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46209 } } }
I20260812 03:48:00.441179 14592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc [sys.catalog]: This master's current role is: LEADER
W20260812 03:48:00.441322 14608 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:48:00.441385 14608 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:48:00.442426 14614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:00.442622 14614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:00.444209 14614 catalog_manager.cc:1347] Generated new cluster ID: 279261563dc74dfb8513cca491233bf5
I20260812 03:48:00.444281 14614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:00.447422 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.434653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.7.132.190:46209)
0812 03:48:00.434848 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:00.434858 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:00.435015 (+ 157us) server_negotiation.cc:407] Connection header received
0812 03:48:00.436235 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:00.436243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:00.436292 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:00.436438 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:00.437306 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.438651 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.439825 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.440515 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.440664 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.443559 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:00.443563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:00.443563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:00.443576 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:00.445342 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.445700 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.445703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.445704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.445794 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.445984 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.445985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.445985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.446332 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:00.446515 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:00.447025 (+ 510us) server_negotiation.cc:299] Negotiation successful
0812 03:48:00.447166 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260812 03:48:00.452147 14614 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:00.452870 14614 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:00.463696 14614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc: Generated new TSK 0
I20260812 03:48:00.463852 14614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:48:00.488943 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46209
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:00.569614 14621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:00.569808 14621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:00.569829 14621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:00.571012 14621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:00.571062 14621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:00.571090 14621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:00.572700 14621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:00.572953 14621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:00.573189 14621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:00.577906 14627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:00.578003 14621 server_base.cc:1034] running on GCE node
W20260812 03:48:00.577986 14629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:00.577906 14632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:00.578380 14621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:00.578599 14621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:00.579738 14621 hybrid_clock.cc:648] HybridClock initialized: now 1786506480579743 us; error 45 us; skew 500 ppm
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:00.582265 14621 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:48:00.582584 14621 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:00.582629 14621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:00.582836 14621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:00.583858 14621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "dacb450d7eac43219a0cddf2fcb630bc"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "72c6064b23ed7a5d7ab894adb06e72f6"
server_key_iv: "3e3c7ce8c6afe15091e2e1b3ef517a80"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.584219 14621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "dacb450d7eac43219a0cddf2fcb630bc"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "72c6064b23ed7a5d7ab894adb06e72f6"
server_key_iv: "3e3c7ce8c6afe15091e2e1b3ef517a80"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.585505 14621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:00.586216 14639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.586414 14621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:00.586514 14621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "dacb450d7eac43219a0cddf2fcb630bc"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "72c6064b23ed7a5d7ab894adb06e72f6"
server_key_iv: "3e3c7ce8c6afe15091e2e1b3ef517a80"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.586606 14621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:00.594528 14621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:00.596839 14621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:00.596997 14621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:00.597245 14621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:00.597558 14621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:00.597612 14621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.597651 14621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:00.597678 14621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.603375 14621 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:36275
I20260812 03:48:00.603474 14822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:36275 every 8 connection(s)
I20260812 03:48:00.603737 14621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:00.612521 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.604446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:46427 (local address 127.7.132.190:46209)
0812 03:48:00.604514 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:00.604517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:00.604755 (+ 238us) server_negotiation.cc:407] Connection header received
0812 03:48:00.605217 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:00.605218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:00.605228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:00.605280 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:00.606162 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.606628 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.607660 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.607864 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.607928 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.609993 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:00.609995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:00.609995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:00.610000 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:00.611399 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.611747 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.611748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.611748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.611800 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.612036 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.612036 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.612036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.612138 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:00.612187 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:00.612398 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 03:48:00.612460 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:48:00.612694 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.604419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46209 (local address 127.7.132.129:46427)
0812 03:48:00.604613 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:00.604647 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:00.605132 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:00.605332 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:00.605335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:00.605587 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:00.606072 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.606077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.606745 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.606746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.607568 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.607570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.607979 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.607980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.608087 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.608566 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:00.608574 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:00.609911 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:00.611469 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:00.611475 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:00.611477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:00.611681 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:00.611872 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:00.611873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:00.611873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:00.611976 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:00.612190 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:00.612194 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:00.612298 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 03:48:00.612373 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:48:00.612477 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":118,"threads_started":1}
I20260812 03:48:00.613279 14824 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46209
I20260812 03:48:00.613358 14824 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:00.613461 14824 heartbeater.cc:511] Master 127.7.132.190:46209 requested a full tablet report, sending...
I20260812 03:48:00.613863 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 14621
I20260812 03:48:00.613970 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:00.614001 14528 ts_manager.cc:194] Registered new tserver with Master: dacb450d7eac43219a0cddf2fcb630bc (127.7.132.129:36275)
I20260812 03:48:00.614168 7698 external_mini_cluster.cc:1428] Setting key 58ec2c6109c750775092be879a4458dc
I20260812 03:48:00.614777 14528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:46427
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:48:00.637310 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46209
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:00.715770 14834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:00.715953 14834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:00.715975 14834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:00.717115 14834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:00.717154 14834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:00.717180 14834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:00.718552 14834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:00.718822 14834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:00.719010 14834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:48:00.723901 14834 server_base.cc:1034] running on GCE node
W20260812 03:48:00.723896 14845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:00.723856 14844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:00.723850 14847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:00.724289 14834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:00.724493 14834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:00.725656 14834 hybrid_clock.cc:648] HybridClock initialized: now 1786506480725638 us; error 40 us; skew 500 ppm
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:00.728083 14834 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:48:00.728434 14834 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:00.728500 14834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:00.728632 14834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:00.729566 14834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "1fc7aa0f2d504e7987cd5e04d490d731"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "31b616fe13d10b6923edfe4990e1c7d4"
server_key_iv: "b5b560e489c217eddfc46aeaff673771"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.729919 14834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "1fc7aa0f2d504e7987cd5e04d490d731"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "31b616fe13d10b6923edfe4990e1c7d4"
server_key_iv: "b5b560e489c217eddfc46aeaff673771"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.731305 14834 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:00.732051 14858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.732216 14834 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:00.732314 14834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "1fc7aa0f2d504e7987cd5e04d490d731"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "31b616fe13d10b6923edfe4990e1c7d4"
server_key_iv: "b5b560e489c217eddfc46aeaff673771"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.732393 14834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:00.744515 14834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:00.746138 14834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:00.746299 14834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:00.746531 14834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:00.746883 14834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:00.746940 14834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.746973 14834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:00.746996 14834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.752687 14834 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:39313
I20260812 03:48:00.752737 15038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:39313 every 8 connection(s)
I20260812 03:48:00.752977 14834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:00.761147 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.753668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:42189 (local address 127.7.132.190:46209)
0812 03:48:00.753712 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:00.753714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:00.753908 (+ 194us) server_negotiation.cc:407] Connection header received
0812 03:48:00.754332 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:00.754333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:00.754342 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:00.754417 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:00.755405 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.755859 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.756719 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.756883 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.756947 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.758858 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:00.758859 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:00.758859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:00.758865 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:00.760098 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.760405 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.760406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.760406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.760450 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.760644 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.760645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.760645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.760758 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:00.760807 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:00.761024 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 03:48:00.761085 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:48:00.761322 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.753603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46209 (local address 127.7.132.130:42189)
0812 03:48:00.753777 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:00.753817 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:00.754253 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:00.754492 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:00.754496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:00.754800 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:00.755315 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.755319 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.755938 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.755938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.756637 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.756638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.756970 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.756970 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.757063 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.757488 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:00.757496 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:00.758784 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:00.760161 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:00.760165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:00.760167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:00.760346 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:00.760506 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:00.760506 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:00.760507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:00.760594 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:00.760813 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:00.760817 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:00.760919 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 03:48:00.760997 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:48:00.761105 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260812 03:48:00.762024 15039 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46209
I20260812 03:48:00.762090 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 14834
I20260812 03:48:00.762131 15039 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:00.762204 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:48:00.762298 15039 heartbeater.cc:511] Master 127.7.132.190:46209 requested a full tablet report, sending...
I20260812 03:48:00.762439 7698 external_mini_cluster.cc:1428] Setting key 1b9c3cd439fb214309c7d463bacbedfe
I20260812 03:48:00.762696 14528 ts_manager.cc:194] Registered new tserver with Master: 1fc7aa0f2d504e7987cd5e04d490d731 (127.7.132.130:39313)
I20260812 03:48:00.763178 14528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:42189
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:48:00.785672 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46209
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:35503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:00.864348 15049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:00.864554 15049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:00.864579 15049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:00.865571 15049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:00.865602 15049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:00.865625 15049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:00.866940 15049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:00.867156 15049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:00.867319 15049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:00.872040 15056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:00.872092 15059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:00.872069 15061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:00.872330 15049 server_base.cc:1034] running on GCE node
I20260812 03:48:00.872541 15049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:00.872736 15049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:00.873878 15049 hybrid_clock.cc:648] HybridClock initialized: now 1786506480873856 us; error 45 us; skew 500 ppm
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:00.876377 15049 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:48:00.876753 15049 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:00.876814 15049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:00.876982 15049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:00.877965 15049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "f2ddd8f3588043a689971ed84e4ebb0d"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "9b7a48488f0f2b3949eaa75143fe1800"
server_key_iv: "bdf80d460bd4e291f1c2e9dace656499"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.878331 15049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "f2ddd8f3588043a689971ed84e4ebb0d"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "9b7a48488f0f2b3949eaa75143fe1800"
server_key_iv: "bdf80d460bd4e291f1c2e9dace656499"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.879673 15049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:00.880472 15071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.880685 15049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:48:00.880792 15049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "f2ddd8f3588043a689971ed84e4ebb0d"
format_stamp: "Formatted at 2026-08-12 03:48:00 on dist-test-slave-39l8"
server_key: "9b7a48488f0f2b3949eaa75143fe1800"
server_key_iv: "bdf80d460bd4e291f1c2e9dace656499"
server_key_version: "encryptionkey@0"
I20260812 03:48:00.880874 15049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:00.897881 15049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:00.899904 15049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:00.900074 15049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:00.900312 15049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:00.900647 15049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:00.900703 15049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.900744 15049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:00.900774 15049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:00.906628 15049 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:40147
I20260812 03:48:00.906679 15258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:40147 every 8 connection(s)
I20260812 03:48:00.906980 15049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:00.910310 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 15049
I20260812 03:48:00.910439 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:00.910650 7698 external_mini_cluster.cc:1428] Setting key b1506262a525011363c08d7b69d4322a
Aug 12 03:48:00 dist-test-slave-39l8 krb5kdc[14456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:00.916450 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.907634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41159 (local address 127.7.132.190:46209)
0812 03:48:00.907677 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:00.907679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:00.907903 (+ 224us) server_negotiation.cc:407] Connection header received
0812 03:48:00.908436 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:00.908438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:00.908452 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:00.908507 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:00.909458 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.909956 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.911361 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.911603 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.911662 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.913784 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:00.913786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:00.913786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:00.913794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:00.915127 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.915513 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.915515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.915515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.915570 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.915826 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.915827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.915827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.915942 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:00.916011 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:00.916279 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 03:48:00.916362 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:48:00.916653 15262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.907628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46209 (local address 127.7.132.131:41159)
0812 03:48:00.907784 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:00.907818 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:00.908313 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:00.908574 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:00.908578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:00.908876 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:00.909355 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.909360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.910073 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.910074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.911270 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.911273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.911682 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.911683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.911833 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.912360 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:00.912368 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:00.913682 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:00.915216 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:00.915223 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:00.915225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:00.915433 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:00.915644 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:00.915645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:00.915645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:00.915752 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:00.916013 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:00.916020 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:00.916151 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 03:48:00.916256 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:48:00.916405 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260812 03:48:00.917183 15259 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46209
I20260812 03:48:00.917268 15259 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:00.917439 15259 heartbeater.cc:511] Master 127.7.132.190:46209 requested a full tablet report, sending...
I20260812 03:48:00.917810 14528 ts_manager.cc:194] Registered new tserver with Master: f2ddd8f3588043a689971ed84e4ebb0d (127.7.132.131:40147)
I20260812 03:48:00.918155 14528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:41159
I20260812 03:48:00.922395 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:48:00.929598 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.923805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.7.132.190:46209)
0812 03:48:00.923860 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:00.923862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:00.923963 (+ 101us) server_negotiation.cc:407] Connection header received
0812 03:48:00.924051 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:00.924052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:00.924069 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:00.924129 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:00.924749 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.925213 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.925927 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.926108 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.926191 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.927320 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:00.927321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:00.927321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:00.927328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:00.928393 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.928716 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.928717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.928717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.928774 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:00.928970 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:00.928971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:00.928971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:00.929105 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:00.929207 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:00.929399 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 03:48:00.929481 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:48:00.931015 14528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:48:00.932552 14528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:00.938586 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.934618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:40147 (local address 127.0.0.1:58976)
0812 03:48:00.934757 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:00.934796 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:00.934903 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:00.935184 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:00.935187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:00.935190 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:00.935255 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.935257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.936394 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.936395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.937390 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.937393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.938273 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.938274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.938352 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.938365 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:48:00.938464 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:48:00.938519 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260812 03:48:00.938598 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.934729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58976 (local address 127.7.132.131:40147)
0812 03:48:00.934955 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:00.934961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:00.934985 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:48:00.935026 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:00.935028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:00.935043 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:00.935179 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:00.935342 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.936296 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.937487 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.938191 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.938270 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.938329 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:00.938482 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 03:48:00.938519 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":127,"threads_started":1}
I20260812 03:48:00.939106 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.935015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:39313 (local address 127.0.0.1:36274)
0812 03:48:00.935089 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:00.935100 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:00.935143 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:00.935672 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:00.935673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:00.935675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:00.935712 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.935713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.936971 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.936972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.938028 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.938031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.938908 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.938908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.938977 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.938981 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:00.939014 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 03:48:00.939045 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":32,"thread_start_us":43,"threads_started":1}
I20260812 03:48:00.939124 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.935333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.7.132.129:36275)
0812 03:48:00.935516 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:00.935521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:00.935531 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:00.935571 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:00.935573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:00.935587 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:00.935672 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:00.935876 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.936888 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.937924 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.938579 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.938647 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.938697 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:00.939010 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 03:48:00.939051 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260812 03:48:00.939244 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.934846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36275 (local address 127.0.0.1:44384)
0812 03:48:00.934955 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:00.934978 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:00.935033 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:00.935772 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:00.935773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:00.935774 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:00.935805 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.935806 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.936988 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.936989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.937839 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:00.937841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.938759 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.938760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:00.938819 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.938822 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:00.938872 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:48:00.938914 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260812 03:48:00.939399 15146 tablet_service.cc:1467] Processing CreateTablet for tablet 881dd61429e54ee0832fdd9d13772b4b (DEFAULT_TABLE table=test-table [id=ac45773402ae4b3e92fea65679114bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:00.939673 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:00.935327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36274 (local address 127.7.132.130:39313)
0812 03:48:00.935516 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:00.935521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:00.935531 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:00.935571 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:00.935573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:00.935587 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:00.935672 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:00.935767 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.936858 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.938118 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:00.938817 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:00.939403 (+ 586us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:00.939514 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:00.939540 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 03:48:00.939584 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260812 03:48:00.939774 15146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 881dd61429e54ee0832fdd9d13772b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:00.939895 14717 tablet_service.cc:1467] Processing CreateTablet for tablet 881dd61429e54ee0832fdd9d13772b4b (DEFAULT_TABLE table=test-table [id=ac45773402ae4b3e92fea65679114bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:00.940234 14717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 881dd61429e54ee0832fdd9d13772b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:00.940276 14928 tablet_service.cc:1467] Processing CreateTablet for tablet 881dd61429e54ee0832fdd9d13772b4b (DEFAULT_TABLE table=test-table [id=ac45773402ae4b3e92fea65679114bb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:00.940568 14928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 881dd61429e54ee0832fdd9d13772b4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:00.943199 15278 tablet_bootstrap.cc:492] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Bootstrap starting.
I20260812 03:48:00.943499 15280 tablet_bootstrap.cc:492] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Bootstrap starting.
I20260812 03:48:00.943830 15278 tablet_bootstrap.cc:654] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:00.943882 15284 tablet_bootstrap.cc:492] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc: Bootstrap starting.
I20260812 03:48:00.944171 15280 tablet_bootstrap.cc:654] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:00.944525 15278 log.cc:824] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:00.944554 15284 tablet_bootstrap.cc:654] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:00.944787 15280 log.cc:824] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:00.945134 15284 log.cc:824] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:00.945315 15278 tablet_bootstrap.cc:492] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: No bootstrap required, opened a new log
I20260812 03:48:00.945403 15278 ts_tablet_manager.cc:1397] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:00.945582 15280 tablet_bootstrap.cc:492] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: No bootstrap required, opened a new log
I20260812 03:48:00.945688 15280 ts_tablet_manager.cc:1397] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:00.946053 15284 tablet_bootstrap.cc:492] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc: No bootstrap required, opened a new log
I20260812 03:48:00.946177 15284 ts_tablet_manager.cc:1397] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:00.947062 15278 raft_consensus.cc:348] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:00.947227 15278 raft_consensus.cc:374] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:00.947261 15278 raft_consensus.cc:729] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ddd8f3588043a689971ed84e4ebb0d, State: Initialized, Role: FOLLOWER
I20260812 03:48:00.947363 15278 consensus_queue.cc:260] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:00.947624 15278 ts_tablet_manager.cc:1428] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:48:00.947593 15280 raft_consensus.cc:348] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:00.947750 15259 heartbeater.cc:503] Master 127.7.132.190:46209 was elected leader, sending a full tablet report...
I20260812 03:48:00.947772 15280 raft_consensus.cc:374] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:00.947814 15280 raft_consensus.cc:729] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fc7aa0f2d504e7987cd5e04d490d731, State: Initialized, Role: FOLLOWER
I20260812 03:48:00.947924 15280 consensus_queue.cc:260] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:00.948185 15039 heartbeater.cc:503] Master 127.7.132.190:46209 was elected leader, sending a full tablet report...
I20260812 03:48:00.948207 15284 raft_consensus.cc:348] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:00.948347 15284 raft_consensus.cc:374] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:00.948381 15284 raft_consensus.cc:729] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dacb450d7eac43219a0cddf2fcb630bc, State: Initialized, Role: FOLLOWER
I20260812 03:48:00.948499 15284 consensus_queue.cc:260] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:00.948554 15280 ts_tablet_manager.cc:1428] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:48:00.948804 15284 ts_tablet_manager.cc:1428] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:48:00.948807 14824 heartbeater.cc:503] Master 127.7.132.190:46209 was elected leader, sending a full tablet report...
W20260812 03:48:01.003571 15040 tablet.cc:2369] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:48:01.104530 14828 tablet.cc:2369] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:48:01.157536 15261 tablet.cc:2369] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:48:01.174067 15288 raft_consensus.cc:482] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:01.174211 15288 raft_consensus.cc:504] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:01.174520 15288 leader_election.cc:288] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dacb450d7eac43219a0cddf2fcb630bc (127.7.132.129:36275), 1fc7aa0f2d504e7987cd5e04d490d731 (127.7.132.130:39313)
I20260812 03:48:01.178369 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.174985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:45177 (local address 127.7.132.130:39313)
0812 03:48:01.175029 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.175031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.175165 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:48:01.175195 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.175196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.175205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.175279 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:01.175433 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.176130 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.177443 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.178004 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.178072 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:01.178081 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.178259 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.178297 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:48:01.178369 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.174823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37767 (local address 127.7.132.129:36275)
0812 03:48:01.174893 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.174895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.174908 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:48:01.175027 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.175028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.175038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.175090 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:01.175264 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.176018 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.177449 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.177992 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.178072 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:01.178081 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.178259 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.178297 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:48:01.178380 15262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.174754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36275 (local address 127.7.132.131:37767)
0812 03:48:01.174832 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.174860 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.174927 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.175106 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.175108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.175110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:01.175183 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:01.175186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.176106 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.176107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:01.177342 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:01.177345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.178090 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.178091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:01.178175 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:01.178177 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.178235 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.178280 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":3}
I20260812 03:48:01.178386 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.174889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:39313 (local address 127.7.132.131:45177)
0812 03:48:01.175080 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.175090 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.175144 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.175299 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.175300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.175302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:01.175369 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:01.175370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.176201 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.176201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:01.177342 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:01.177345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.178090 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.178091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:01.178170 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:01.178175 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.178245 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.178280 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":7,"thread_start_us":129,"threads_started":1}
I20260812 03:48:01.178866 14754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "881dd61429e54ee0832fdd9d13772b4b" candidate_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dacb450d7eac43219a0cddf2fcb630bc" is_pre_election: true
I20260812 03:48:01.179032 14754 raft_consensus.cc:2454] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ddd8f3588043a689971ed84e4ebb0d in term 0.
I20260812 03:48:01.179003 14961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "881dd61429e54ee0832fdd9d13772b4b" candidate_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" is_pre_election: true
I20260812 03:48:01.179174 14961 raft_consensus.cc:2454] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ddd8f3588043a689971ed84e4ebb0d in term 0.
I20260812 03:48:01.179252 15077 leader_election.cc:302] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dacb450d7eac43219a0cddf2fcb630bc, f2ddd8f3588043a689971ed84e4ebb0d; no voters:
I20260812 03:48:01.179407 15288 raft_consensus.cc:2781] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:01.179482 15288 raft_consensus.cc:482] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:01.179500 15288 raft_consensus.cc:3037] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:01.180240 15288 raft_consensus.cc:504] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:01.180379 15288 leader_election.cc:288] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [CANDIDATE]: Term 1 election: Requested vote from peers dacb450d7eac43219a0cddf2fcb630bc (127.7.132.129:36275), 1fc7aa0f2d504e7987cd5e04d490d731 (127.7.132.130:39313)
I20260812 03:48:01.180542 14754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "881dd61429e54ee0832fdd9d13772b4b" candidate_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dacb450d7eac43219a0cddf2fcb630bc"
I20260812 03:48:01.180586 14961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "881dd61429e54ee0832fdd9d13772b4b" candidate_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fc7aa0f2d504e7987cd5e04d490d731"
I20260812 03:48:01.180622 14754 raft_consensus.cc:3037] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:01.180668 14961 raft_consensus.cc:3037] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:01.181321 14754 raft_consensus.cc:2454] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ddd8f3588043a689971ed84e4ebb0d in term 1.
I20260812 03:48:01.181423 14961 raft_consensus.cc:2454] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ddd8f3588043a689971ed84e4ebb0d in term 1.
I20260812 03:48:01.181497 15077 leader_election.cc:302] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dacb450d7eac43219a0cddf2fcb630bc, f2ddd8f3588043a689971ed84e4ebb0d; no voters:
I20260812 03:48:01.181622 15288 raft_consensus.cc:2781] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:01.181802 15288 raft_consensus.cc:686] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 1 LEADER]: Becoming Leader. State: Replica: f2ddd8f3588043a689971ed84e4ebb0d, State: Running, Role: LEADER
I20260812 03:48:01.181921 15288 consensus_queue.cc:237] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } }
I20260812 03:48:01.182802 14523 catalog_manager.cc:5495] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d reported cstate change: term changed from 0 to 1, leader changed from <none> to f2ddd8f3588043a689971ed84e4ebb0d (127.7.132.131). New cstate: current_term: 1 leader_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ddd8f3588043a689971ed84e4ebb0d" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 } health_report { overall_health: UNKNOWN } } }
I20260812 03:48:01.220516 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.218294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58980 (local address 127.7.132.131:40147)
0812 03:48:01.218356 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.218358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.218384 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:48:01.218420 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.218421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.218441 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.218496 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:01.218632 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.219107 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.219819 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:01.219998 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:01.220045 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:01.220182 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:01.220279 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:01.220339 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.220370 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.220389 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:48:01.222522 14754 raft_consensus.cc:1261] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc [term 1 FOLLOWER]: Refusing update from remote peer f2ddd8f3588043a689971ed84e4ebb0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:01.222522 14961 raft_consensus.cc:1261] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 1 FOLLOWER]: Refusing update from remote peer f2ddd8f3588043a689971ed84e4ebb0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:01.222903 15295 consensus_queue.cc:1035] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fc7aa0f2d504e7987cd5e04d490d731" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:01.223085 15288 consensus_queue.cc:1035] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "dacb450d7eac43219a0cddf2fcb630bc" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 36275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:01.226310 15303 mvcc.cc:205] Tried to move back new op lower bound from 7317530547084292096 to 7317530546921758720. Current Snapshot: MvccSnapshot[applied={T|T < 7317530547084292096}]
I20260812 03:48:01.227674 14523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:01.227773 14523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:01.228536 14523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 753dc33555fc40c8bdd80487ee307bbc: Sending DeleteTablet for 3 replicas of tablet 881dd61429e54ee0832fdd9d13772b4b
I20260812 03:48:01.228844 15146 tablet_service.cc:1514] Processing DeleteTablet for tablet 881dd61429e54ee0832fdd9d13772b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:01 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:58976
I20260812 03:48:01.228892 14717 tablet_service.cc:1514] Processing DeleteTablet for tablet 881dd61429e54ee0832fdd9d13772b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:01 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:44384
I20260812 03:48:01.228955 14928 tablet_service.cc:1514] Processing DeleteTablet for tablet 881dd61429e54ee0832fdd9d13772b4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:01 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36274
I20260812 03:48:01.229167 15320 tablet_replica.cc:330] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: stopping tablet replica
I20260812 03:48:01.229310 15320 raft_consensus.cc:2229] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:01.229394 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 14621
I20260812 03:48:01.229439 15319 tablet_replica.cc:330] T 881dd61429e54ee0832fdd9d13772b4b P dacb450d7eac43219a0cddf2fcb630bc: stopping tablet replica
I20260812 03:48:01.237547 15318 tablet_replica.cc:330] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: stopping tablet replica
I20260812 03:48:01.237663 15318 raft_consensus.cc:2229] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:01.237776 15318 raft_consensus.cc:2258] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:01.237895 15320 raft_consensus.cc:2258] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:01.238268 15318 ts_tablet_manager.cc:1905] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:01.238399 15320 ts_tablet_manager.cc:1905] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:01.239985 15318 ts_tablet_manager.cc:1918] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:01.240068 15318 log.cc:1196] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/881dd61429e54ee0832fdd9d13772b4b
I20260812 03:48:01.240410 15320 ts_tablet_manager.cc:1918] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:01.240424 15318 ts_tablet_manager.cc:1939] T 881dd61429e54ee0832fdd9d13772b4b P f2ddd8f3588043a689971ed84e4ebb0d: Deleting consensus metadata
I20260812 03:48:01.240487 15320 log.cc:1196] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/881dd61429e54ee0832fdd9d13772b4b
I20260812 03:48:01.240782 15320 ts_tablet_manager.cc:1939] T 881dd61429e54ee0832fdd9d13772b4b P 1fc7aa0f2d504e7987cd5e04d490d731: Deleting consensus metadata
I20260812 03:48:01.241256 14506 catalog_manager.cc:4841] TS f2ddd8f3588043a689971ed84e4ebb0d (127.7.132.131:40147): tablet 881dd61429e54ee0832fdd9d13772b4b (table test-table [id=ac45773402ae4b3e92fea65679114bb3]) successfully deleted
I20260812 03:48:01.241372 14506 catalog_manager.cc:4841] TS 1fc7aa0f2d504e7987cd5e04d490d731 (127.7.132.130:39313): tablet 881dd61429e54ee0832fdd9d13772b4b (table test-table [id=ac45773402ae4b3e92fea65679114bb3]) successfully deleted
W20260812 03:48:01.243870 14506 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:36275 (error 108)
I20260812 03:48:01.244208 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 14834
I20260812 03:48:01.244356 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.244218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:36275 (local address 127.0.0.1:44400)
0812 03:48:01.244283 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.244325 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 03:48:01.248128 14506 catalog_manager.cc:4584] TS dacb450d7eac43219a0cddf2fcb630bc (127.7.132.129:36275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 881dd61429e54ee0832fdd9d13772b4b: Network error: Client connection negotiation failed: client connection to 127.7.132.129:36275: connect: Connection refused (error 111)
I20260812 03:48:01.253780 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 15049
I20260812 03:48:01.262128 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 14484
2026-08-12T03:48:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2292 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T03:48:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:01Z Disabled control of system clock
I20260812 03:48:01.293030 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46375
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:48:01.373474 15328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:01.373656 15328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:01.373677 15328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:01.374794 15328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:01.374831 15328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:01.374845 15328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:01.374881 15328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:01.374907 15328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:01.376209 15328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:01.376459 15328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:01.376757 15328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:01.381289 15340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:01.381338 15341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:01.381300 15345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:01.381547 15328 server_base.cc:1034] running on GCE node
I20260812 03:48:01.381798 15328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:01.382040 15328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:01.383176 15328 hybrid_clock.cc:648] HybridClock initialized: now 1786506481383166 us; error 28 us; skew 500 ppm
I20260812 03:48:01.384377 15328 webserver.cc:460] Webserver started at http://127.7.132.190:46779/ using document root <none> and password file <none>
I20260812 03:48:01.384629 15328 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:01.384698 15328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:01.384872 15328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:01.385874 15328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "ade8873c3b53403b9bfc516c7578a592"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "30e02dca6c9ba82a2bbd77642ab6040b"
server_key_iv: "754ed3d828afbb4bf9e6a00d33638e54"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.386222 15328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "ade8873c3b53403b9bfc516c7578a592"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "30e02dca6c9ba82a2bbd77642ab6040b"
server_key_iv: "754ed3d828afbb4bf9e6a00d33638e54"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.387631 15328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:01.388381 15357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.388556 15328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:01.388660 15328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "ade8873c3b53403b9bfc516c7578a592"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "30e02dca6c9ba82a2bbd77642ab6040b"
server_key_iv: "754ed3d828afbb4bf9e6a00d33638e54"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.388744 15328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:01.392057 15328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:01.392323 15328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:01.392468 15328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:01.396525 15328 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46375
I20260812 03:48:01.396555 15438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46375 every 8 connection(s)
I20260812 03:48:01.396839 15328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:01.397250 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 15328
I20260812 03:48:01.397373 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:01.397537 7698 external_mini_cluster.cc:1428] Setting key 1aca07e046b1820001975d4e009c2e21
I20260812 03:48:01.397572 15439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:01.400256 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.398237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37870 (local address 127.7.132.190:46375)
0812 03:48:01.398413 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.398420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.398442 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:48:01.399069 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.399076 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.399115 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.399220 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.399867 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.399870 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.399870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.399874 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.399909 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.399918 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.399971 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.400065 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.400108 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260812 03:48:01.400998 15439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592: Bootstrap starting.
I20260812 03:48:01.401548 15439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:01.401784 15439 log.cc:824] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:01.402429 15439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592: No bootstrap required, opened a new log
I20260812 03:48:01.403776 15439 raft_consensus.cc:348] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8873c3b53403b9bfc516c7578a592" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46375 } }
I20260812 03:48:01.403896 15439 raft_consensus.cc:374] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:01.403918 15439 raft_consensus.cc:729] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade8873c3b53403b9bfc516c7578a592, State: Initialized, Role: FOLLOWER
I20260812 03:48:01.404022 15439 consensus_queue.cc:260] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8873c3b53403b9bfc516c7578a592" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46375 } }
I20260812 03:48:01.404106 15439 raft_consensus.cc:388] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:01.404145 15439 raft_consensus.cc:482] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:01.404213 15439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:01.404721 15439 raft_consensus.cc:504] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8873c3b53403b9bfc516c7578a592" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46375 } }
I20260812 03:48:01.404858 15439 leader_election.cc:302] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ade8873c3b53403b9bfc516c7578a592; no voters:
I20260812 03:48:01.405037 15439 leader_election.cc:288] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:01.405303 15439 sys_catalog.cc:564] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:01.405313 15447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:01.405488 15447 raft_consensus.cc:686] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [term 1 LEADER]: Becoming Leader. State: Replica: ade8873c3b53403b9bfc516c7578a592, State: Running, Role: LEADER
I20260812 03:48:01.405577 15447 consensus_queue.cc:237] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8873c3b53403b9bfc516c7578a592" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46375 } }
I20260812 03:48:01.405936 15448 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ade8873c3b53403b9bfc516c7578a592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8873c3b53403b9bfc516c7578a592" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46375 } } }
I20260812 03:48:01.405951 15450 sys_catalog.cc:455] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ade8873c3b53403b9bfc516c7578a592. Latest consensus state: current_term: 1 leader_uuid: "ade8873c3b53403b9bfc516c7578a592" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ade8873c3b53403b9bfc516c7578a592" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46375 } } }
I20260812 03:48:01.406082 15448 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:01.406090 15450 sys_catalog.cc:458] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:01.406340 15471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:01.406831 15471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:01.408296 15471 catalog_manager.cc:1347] Generated new cluster ID: 3eab70d18e88463087189cb5b01a5649
I20260812 03:48:01.408352 15471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:01.420249 15471 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:01.420415 15471 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:01.427269 15471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592: Generated new TSK 0
I20260812 03:48:01.427435 15471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:48:01.430523 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46375
--builtin_ntp_servers=127.7.132.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:48:01.525220 15485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:01.525415 15485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:01.525436 15485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:01.526538 15485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:01.526576 15485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:01.526599 15485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:01.527947 15485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:01.528185 15485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:01.528481 15485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:01.534322 15493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:01.534308 15492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:01.534533 15485 server_base.cc:1034] running on GCE node
W20260812 03:48:01.534577 15495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:01.534837 15485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:01.535081 15485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:01.536244 15485 hybrid_clock.cc:648] HybridClock initialized: now 1786506481536231 us; error 31 us; skew 500 ppm
I20260812 03:48:01.537575 15485 webserver.cc:460] Webserver started at http://127.7.132.129:38695/ using document root <none> and password file <none>
I20260812 03:48:01.537823 15485 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:01.537892 15485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:01.538051 15485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:01.539187 15485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "cf72bc3e99654df4817ae33ae5d24c8b"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "a40c44176b9b0812e9e294b6858df5b2"
server_key_iv: "00d1b79f325aed0757aada8e93186e57"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.539556 15485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "cf72bc3e99654df4817ae33ae5d24c8b"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "a40c44176b9b0812e9e294b6858df5b2"
server_key_iv: "00d1b79f325aed0757aada8e93186e57"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.540954 15485 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:48:01.541749 15503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.541944 15485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:01.542047 15485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "cf72bc3e99654df4817ae33ae5d24c8b"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "a40c44176b9b0812e9e294b6858df5b2"
server_key_iv: "00d1b79f325aed0757aada8e93186e57"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.542132 15485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:01.545380 15485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:01.545639 15485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:01.545787 15485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:01.546006 15485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:01.546295 15485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:01.546347 15485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.546384 15485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:01.546408 15485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.552369 15485 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:38521
I20260812 03:48:01.552448 15684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:38521 every 8 connection(s)
I20260812 03:48:01.552795 15485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:48:01.555871 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.553591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:51455 (local address 127.7.132.190:46375)
0812 03:48:01.553671 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.553673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.553891 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:48:01.554359 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.554361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.554367 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.554443 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.555634 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.555636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.555636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.555641 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.555659 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.555663 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.555732 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.555782 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.555798 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:48:01.556025 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.553574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46375 (local address 127.7.132.129:51455)
0812 03:48:01.553760 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.553799 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.554264 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.554502 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.554506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.554513 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:01.555484 (+ 971us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:01.555484 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:01.555512 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:01.555538 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:01.555730 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:01.555732 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:01.555735 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.555785 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.555830 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260812 03:48:01.556504 15687 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46375
I20260812 03:48:01.556618 15687 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:01.556780 15687 heartbeater.cc:511] Master 127.7.132.190:46375 requested a full tablet report, sending...
I20260812 03:48:01.556772 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 15485
I20260812 03:48:01.556876 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:01.557062 7698 external_mini_cluster.cc:1428] Setting key 8e266e3d41b12238c3c8be9cafa7df98
I20260812 03:48:01.557336 15384 ts_manager.cc:194] Registered new tserver with Master: cf72bc3e99654df4817ae33ae5d24c8b (127.7.132.129:38521)
I20260812 03:48:01.558111 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46375
--builtin_ntp_servers=127.7.132.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:48:01.638845 15691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:01.639027 15691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:01.639048 15691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:01.640071 15691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:01.640105 15691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:01.640128 15691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:01.641392 15691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:01.641564 15691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:01.641790 15691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:01.646669 15702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:01.646682 15696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:01.646685 15699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:01.646886 15691 server_base.cc:1034] running on GCE node
I20260812 03:48:01.647070 15691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:01.647320 15691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:01.648481 15691 hybrid_clock.cc:648] HybridClock initialized: now 1786506481648468 us; error 42 us; skew 500 ppm
I20260812 03:48:01.649662 15691 webserver.cc:460] Webserver started at http://127.7.132.130:37595/ using document root <none> and password file <none>
I20260812 03:48:01.649907 15691 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:01.649976 15691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:01.650110 15691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:01.651198 15691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "edf73cc1b3324b8caf421aeae37902e6"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "802338a41f47b845415fea92a9775c5c"
server_key_iv: "32a2f9da23610db8fc5c1e11f497f337"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.651568 15691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "edf73cc1b3324b8caf421aeae37902e6"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "802338a41f47b845415fea92a9775c5c"
server_key_iv: "32a2f9da23610db8fc5c1e11f497f337"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.652871 15691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:48:01.653599 15712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.653769 15691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:01.653869 15691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "edf73cc1b3324b8caf421aeae37902e6"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "802338a41f47b845415fea92a9775c5c"
server_key_iv: "32a2f9da23610db8fc5c1e11f497f337"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.653949 15691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:01.657188 15691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:01.657430 15691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:01.657562 15691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:01.657768 15691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:01.658070 15691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:01.658120 15691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.658157 15691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:01.658186 15691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.663895 15691 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:41517
I20260812 03:48:01.663928 15895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:41517 every 8 connection(s)
I20260812 03:48:01.664234 15691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:01.667038 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.664890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:51071 (local address 127.7.132.190:46375)
0812 03:48:01.664932 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.664934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.665137 (+ 203us) server_negotiation.cc:407] Connection header received
0812 03:48:01.665641 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.665642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.665658 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.665712 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.666799 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.666801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.666802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.666807 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.666822 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.666826 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.666885 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.666953 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.666968 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:48:01.667246 15899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.664860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46375 (local address 127.7.132.130:51071)
0812 03:48:01.665025 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.665055 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.665523 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.665778 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.665782 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.665791 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:01.666626 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:01.666626 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:01.666654 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:01.666679 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:01.666885 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:01.666891 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:01.666894 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.666966 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.667023 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260812 03:48:01.667703 15896 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46375
I20260812 03:48:01.667809 15896 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:01.667941 15896 heartbeater.cc:511] Master 127.7.132.190:46375 requested a full tablet report, sending...
I20260812 03:48:01.668318 15383 ts_manager.cc:194] Registered new tserver with Master: edf73cc1b3324b8caf421aeae37902e6 (127.7.132.130:41517)
I20260812 03:48:01.673466 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 15691
I20260812 03:48:01.673573 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:48:01.673763 7698 external_mini_cluster.cc:1428] Setting key aa09128e356d926f6b75c0b8835d7676
I20260812 03:48:01.674567 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46375
--builtin_ntp_servers=127.7.132.148:42699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:48:01.752427 15900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:01.752650 15900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:01.752687 15900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:01.753773 15900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:01.753831 15900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:01.753868 15900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:01.755090 15900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:01.755313 15900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:01.755554 15900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:01.760281 15909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:01.760340 15907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:01.760340 15911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:01.760413 15900 server_base.cc:1034] running on GCE node
I20260812 03:48:01.760696 15900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:01.760951 15900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:01.762105 15900 hybrid_clock.cc:648] HybridClock initialized: now 1786506481762085 us; error 36 us; skew 500 ppm
I20260812 03:48:01.763366 15900 webserver.cc:460] Webserver started at http://127.7.132.131:42337/ using document root <none> and password file <none>
I20260812 03:48:01.763587 15900 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:01.763660 15900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:01.763808 15900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:01.764809 15900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "aaa156388a4e474780b0b0734db309cc"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "d2f2dc55088ffd5fce70bef9c8a63be7"
server_key_iv: "b1d40dafa8bef5b15a4a69d7ad81ef4f"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.765216 15900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "aaa156388a4e474780b0b0734db309cc"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "d2f2dc55088ffd5fce70bef9c8a63be7"
server_key_iv: "b1d40dafa8bef5b15a4a69d7ad81ef4f"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.766489 15900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:01.767289 15917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.767438 15900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.767530 15900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "aaa156388a4e474780b0b0734db309cc"
format_stamp: "Formatted at 2026-08-12 03:48:01 on dist-test-slave-39l8"
server_key: "d2f2dc55088ffd5fce70bef9c8a63be7"
server_key_iv: "b1d40dafa8bef5b15a4a69d7ad81ef4f"
server_key_version: "encryptionkey@0"
I20260812 03:48:01.767602 15900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:01.770859 15900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:01.771103 15900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:01.771237 15900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:01.771449 15900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:01.771749 15900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:01.771801 15900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.771842 15900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:01.771868 15900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:01.777206 15900 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:39323
I20260812 03:48:01.777240 16102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:39323 every 8 connection(s)
I20260812 03:48:01.777534 15900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:01.779843 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 15900
I20260812 03:48:01.779932 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:01.780118 7698 external_mini_cluster.cc:1428] Setting key f8d8f67f22a5d775e45a94d3e28c11cd
I20260812 03:48:01.780598 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.778227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:53557 (local address 127.7.132.190:46375)
0812 03:48:01.778292 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.778294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.778496 (+ 202us) server_negotiation.cc:407] Connection header received
0812 03:48:01.778987 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.778988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.778994 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.779049 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.780283 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.780284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.780285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.780290 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.780332 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.780336 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.780391 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.780515 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.780530 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:48:01.780859 16107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.778209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46375 (local address 127.7.132.131:53557)
0812 03:48:01.778394 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.778428 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.778900 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.779121 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.779124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.779131 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:01.780123 (+ 992us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:01.780123 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:01.780167 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:01.780201 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:01.780411 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:01.780418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:01.780421 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.780510 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.780578 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260812 03:48:01.781324 16104 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46375
I20260812 03:48:01.781412 16104 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:01.781574 16104 heartbeater.cc:511] Master 127.7.132.190:46375 requested a full tablet report, sending...
I20260812 03:48:01.781956 15384 ts_manager.cc:194] Registered new tserver with Master: aaa156388a4e474780b0b0734db309cc (127.7.132.131:39323)
I20260812 03:48:01.792111 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:48:01.794318 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.793401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.7.132.190:46375)
0812 03:48:01.793447 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.793448 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.793541 (+ 93us) server_negotiation.cc:407] Connection header received
0812 03:48:01.793588 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.793589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.793592 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.793636 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.794190 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.794191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.794191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.794194 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.794202 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.794205 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.794239 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.794267 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.794276 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:48:01.795953 15384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:48:01.797433 15384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:01.801358 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.799550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.7.132.131:39323)
0812 03:48:01.799842 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.799847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.799857 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:01.799880 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.799882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.799889 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.799979 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.800818 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.800826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.800827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.800831 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.800864 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.800886 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.800943 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.801259 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.801277 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":84,"threads_started":1}
I20260812 03:48:01.801343 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.799521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:41517 (local address 127.0.0.1:55740)
0812 03:48:01.799910 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.799921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.799975 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.800467 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.800468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.800471 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:01.800899 (+ 428us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:01.800899 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:01.800906 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:01.800919 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:01.801167 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:01.801168 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:01.801169 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.801244 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.801258 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":106,"threads_started":1}
I20260812 03:48:01.801527 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.799891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.7.132.129:38521)
0812 03:48:01.800102 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.800106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.800428 (+ 322us) server_negotiation.cc:407] Connection header received
0812 03:48:01.800466 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.800469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.800479 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.800804 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.801185 (+ 381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.801192 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.801193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.801197 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.801226 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.801250 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.801307 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.801422 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.801438 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260812 03:48:01.801640 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.799637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55740 (local address 127.7.132.130:41517)
0812 03:48:01.799809 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.799814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.800032 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:48:01.800063 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.800065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.800075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.800158 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.801006 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.801013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.801013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.801019 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.801055 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.801084 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.801142 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.801498 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.801517 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260812 03:48:01.801350 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.799483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39323 (local address 127.0.0.1:51018)
0812 03:48:01.799714 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.799756 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.799814 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.799983 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.799990 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.799996 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:01.800697 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:01.800698 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:01.800713 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:01.800733 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:01.800992 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:01.800999 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:01.801002 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.801233 (+ 231us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.801247 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":62,"threads_started":1}
I20260812 03:48:01.801473 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.799679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:38521 (local address 127.0.0.1:37592)
0812 03:48:01.800352 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.800363 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.800416 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.800561 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.800562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.800565 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:01.801092 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:01.801092 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:01.801098 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:01.801109 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:01.801333 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:01.801333 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:01.801334 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.801377 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.801390 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":7,"thread_start_us":76,"threads_started":1}
I20260812 03:48:01.802376 15793 tablet_service.cc:1467] Processing CreateTablet for tablet 2336ff2466ed4a7caac0e374e46df656 (DEFAULT_TABLE table=test-table [id=4e5fe14f70d14af3a33e402c63b18818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:01.802405 15991 tablet_service.cc:1467] Processing CreateTablet for tablet 2336ff2466ed4a7caac0e374e46df656 (DEFAULT_TABLE table=test-table [id=4e5fe14f70d14af3a33e402c63b18818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:01.802407 15577 tablet_service.cc:1467] Processing CreateTablet for tablet 2336ff2466ed4a7caac0e374e46df656 (DEFAULT_TABLE table=test-table [id=4e5fe14f70d14af3a33e402c63b18818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:01.802752 15577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2336ff2466ed4a7caac0e374e46df656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:01.802764 15991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2336ff2466ed4a7caac0e374e46df656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:01.802794 15793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2336ff2466ed4a7caac0e374e46df656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:01.805840 16124 tablet_bootstrap.cc:492] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Bootstrap starting.
I20260812 03:48:01.806027 16125 tablet_bootstrap.cc:492] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: Bootstrap starting.
I20260812 03:48:01.806161 16128 tablet_bootstrap.cc:492] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b: Bootstrap starting.
I20260812 03:48:01.806520 16124 tablet_bootstrap.cc:654] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:01.806726 16125 tablet_bootstrap.cc:654] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:01.806835 16128 tablet_bootstrap.cc:654] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:01.806865 16124 log.cc:824] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:01.807067 16125 log.cc:824] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:01.807142 16128 log.cc:824] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:01.807606 16124 tablet_bootstrap.cc:492] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: No bootstrap required, opened a new log
I20260812 03:48:01.807777 16125 tablet_bootstrap.cc:492] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: No bootstrap required, opened a new log
I20260812 03:48:01.807807 16124 ts_tablet_manager.cc:1397] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:48:01.807848 16125 ts_tablet_manager.cc:1397] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:48:01.808000 16128 tablet_bootstrap.cc:492] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b: No bootstrap required, opened a new log
I20260812 03:48:01.808063 16128 ts_tablet_manager.cc:1397] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:48:01.809545 16124 raft_consensus.cc:348] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.809696 16124 raft_consensus.cc:374] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:01.809738 16124 raft_consensus.cc:729] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf73cc1b3324b8caf421aeae37902e6, State: Initialized, Role: FOLLOWER
I20260812 03:48:01.809844 16124 consensus_queue.cc:260] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.810113 16124 ts_tablet_manager.cc:1428] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:48:01.810122 15896 heartbeater.cc:503] Master 127.7.132.190:46375 was elected leader, sending a full tablet report...
I20260812 03:48:01.810266 16125 raft_consensus.cc:348] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.810420 16125 raft_consensus.cc:374] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:01.810464 16125 raft_consensus.cc:729] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa156388a4e474780b0b0734db309cc, State: Initialized, Role: FOLLOWER
I20260812 03:48:01.810571 16125 consensus_queue.cc:260] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.810364 16128 raft_consensus.cc:348] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.810654 16128 raft_consensus.cc:374] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:01.810745 16128 raft_consensus.cc:729] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf72bc3e99654df4817ae33ae5d24c8b, State: Initialized, Role: FOLLOWER
I20260812 03:48:01.810878 16125 ts_tablet_manager.cc:1428] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260812 03:48:01.810873 16128 consensus_queue.cc:260] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.810993 16104 heartbeater.cc:503] Master 127.7.132.190:46375 was elected leader, sending a full tablet report...
I20260812 03:48:01.811121 16128 ts_tablet_manager.cc:1428] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 03:48:01.811283 15687 heartbeater.cc:503] Master 127.7.132.190:46375 was elected leader, sending a full tablet report...
I20260812 03:48:01.907366 16138 raft_consensus.cc:482] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:01.907485 16138 raft_consensus.cc:504] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.907893 16138 leader_election.cc:288] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaa156388a4e474780b0b0734db309cc (127.7.132.131:39323), edf73cc1b3324b8caf421aeae37902e6 (127.7.132.130:41517)
I20260812 03:48:01.909288 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.908199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:39383 (local address 127.7.132.130:41517)
0812 03:48:01.908249 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.908251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.908276 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:48:01.908337 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.908338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.908345 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.908400 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.909056 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.909057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.909058 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.909063 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.909079 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.909084 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.909143 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.909207 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.909220 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:48:01.909288 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.908101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:42295 (local address 127.7.132.131:39323)
0812 03:48:01.908152 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.908155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.908331 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:48:01.908364 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.908365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.908370 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.908423 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.909032 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.909033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.909034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.909039 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.909057 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.909062 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.909125 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.909198 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.909213 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:48:01.909319 15690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.908047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:41517 (local address 127.7.132.129:39383)
0812 03:48:01.908188 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.908200 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.908243 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.908419 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.908420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.908422 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:01.908961 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:01.908962 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:01.908966 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:01.908976 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:01.909145 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:01.909146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:01.909147 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.909204 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.909229 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":14}
I20260812 03:48:01.909330 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.908067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:39323 (local address 127.7.132.129:42295)
0812 03:48:01.908239 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.908248 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:01.908293 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:01.908425 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:01.908426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:01.908444 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:01.908894 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:01.908895 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:01.908903 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:01.908932 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:01.909145 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:01.909146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:01.909147 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:48:01.909199 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:48:01.909215 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
I20260812 03:48:01.909708 15824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2336ff2466ed4a7caac0e374e46df656" candidate_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf73cc1b3324b8caf421aeae37902e6" is_pre_election: true
I20260812 03:48:01.909752 16023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2336ff2466ed4a7caac0e374e46df656" candidate_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa156388a4e474780b0b0734db309cc" is_pre_election: true
I20260812 03:48:01.909873 16023 raft_consensus.cc:2454] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf72bc3e99654df4817ae33ae5d24c8b in term 0.
I20260812 03:48:01.909873 15824 raft_consensus.cc:2454] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf72bc3e99654df4817ae33ae5d24c8b in term 0.
I20260812 03:48:01.910101 15512 leader_election.cc:302] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa156388a4e474780b0b0734db309cc, cf72bc3e99654df4817ae33ae5d24c8b; no voters:
I20260812 03:48:01.910240 16138 raft_consensus.cc:2781] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:01.910295 16138 raft_consensus.cc:482] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:01.910316 16138 raft_consensus.cc:3037] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:01.911088 16138 raft_consensus.cc:504] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.911249 16138 leader_election.cc:288] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [CANDIDATE]: Term 1 election: Requested vote from peers aaa156388a4e474780b0b0734db309cc (127.7.132.131:39323), edf73cc1b3324b8caf421aeae37902e6 (127.7.132.130:41517)
I20260812 03:48:01.911456 16023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2336ff2466ed4a7caac0e374e46df656" candidate_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaa156388a4e474780b0b0734db309cc"
I20260812 03:48:01.911530 16023 raft_consensus.cc:3037] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:01.912307 16023 raft_consensus.cc:2454] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf72bc3e99654df4817ae33ae5d24c8b in term 1.
I20260812 03:48:01.912434 15824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2336ff2466ed4a7caac0e374e46df656" candidate_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf73cc1b3324b8caf421aeae37902e6"
I20260812 03:48:01.912478 15512 leader_election.cc:302] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaa156388a4e474780b0b0734db309cc, cf72bc3e99654df4817ae33ae5d24c8b; no voters:
I20260812 03:48:01.912524 15824 raft_consensus.cc:3037] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:01.912590 16138 raft_consensus.cc:2781] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:01.912744 16138 raft_consensus.cc:686] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 1 LEADER]: Becoming Leader. State: Replica: cf72bc3e99654df4817ae33ae5d24c8b, State: Running, Role: LEADER
I20260812 03:48:01.912945 16138 consensus_queue.cc:237] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } }
I20260812 03:48:01.913403 15824 raft_consensus.cc:2454] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf72bc3e99654df4817ae33ae5d24c8b in term 1.
I20260812 03:48:01.913883 15384 catalog_manager.cc:5495] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b reported cstate change: term changed from 0 to 1, leader changed from <none> to cf72bc3e99654df4817ae33ae5d24c8b (127.7.132.129). New cstate: current_term: 1 leader_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf72bc3e99654df4817ae33ae5d24c8b" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 38521 } health_report { overall_health: HEALTHY } } }
W20260812 03:48:01.914741 15897 tablet.cc:2369] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:48:01.919092 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.918047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37598 (local address 127.7.132.129:38521)
0812 03:48:01.918116 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:01.918118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:01.918127 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:01.918214 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:01.918215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:01.918227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:01.918276 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:01.918882 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:01.918883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:01.918883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:01.918888 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:01.918902 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:01.918907 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:01.918964 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:01.919015 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:48:01.919027 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:48:01.920607 15824 raft_consensus.cc:1261] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 1 FOLLOWER]: Refusing update from remote peer cf72bc3e99654df4817ae33ae5d24c8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:01.920670 16023 raft_consensus.cc:1261] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 1 FOLLOWER]: Refusing update from remote peer cf72bc3e99654df4817ae33ae5d24c8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:01.920938 16138 consensus_queue.cc:1035] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf73cc1b3324b8caf421aeae37902e6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 41517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:01.921103 16140 consensus_queue.cc:1035] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaa156388a4e474780b0b0734db309cc" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 39323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:01.925583 15380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:01.925694 15380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:01.926429 15380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ade8873c3b53403b9bfc516c7578a592: Sending DeleteTablet for 3 replicas of tablet 2336ff2466ed4a7caac0e374e46df656
I20260812 03:48:01.926744 15991 tablet_service.cc:1514] Processing DeleteTablet for tablet 2336ff2466ed4a7caac0e374e46df656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:01 UTC) from {username='slave'} at 127.0.0.1:51018
I20260812 03:48:01.926800 15577 tablet_service.cc:1514] Processing DeleteTablet for tablet 2336ff2466ed4a7caac0e374e46df656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:01 UTC) from {username='slave'} at 127.0.0.1:37592
I20260812 03:48:01.926795 15793 tablet_service.cc:1514] Processing DeleteTablet for tablet 2336ff2466ed4a7caac0e374e46df656 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:01 UTC) from {username='slave'} at 127.0.0.1:55740
I20260812 03:48:01.927047 16168 tablet_replica.cc:330] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: stopping tablet replica
I20260812 03:48:01.927045 16164 tablet_replica.cc:330] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b: stopping tablet replica
I20260812 03:48:01.927140 16168 raft_consensus.cc:2229] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:01.927139 16164 raft_consensus.cc:2229] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:01.927232 16168 raft_consensus.cc:2258] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:01.927258 16164 raft_consensus.cc:2258] T 2336ff2466ed4a7caac0e374e46df656 P cf72bc3e99654df4817ae33ae5d24c8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:01.927407 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 15485
I20260812 03:48:01.946902 16168 ts_tablet_manager.cc:1905] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:01.948190 16162 tablet_replica.cc:330] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: stopping tablet replica
I20260812 03:48:01.948336 16162 raft_consensus.cc:2229] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:01.948463 16162 raft_consensus.cc:2258] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:01.948951 16168 ts_tablet_manager.cc:1918] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:01.949080 16168 log.cc:1196] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/2336ff2466ed4a7caac0e374e46df656
I20260812 03:48:01.949431 16168 ts_tablet_manager.cc:1939] T 2336ff2466ed4a7caac0e374e46df656 P edf73cc1b3324b8caf421aeae37902e6: Deleting consensus metadata
I20260812 03:48:01.949717 16162 ts_tablet_manager.cc:1905] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:01.949965 15359 catalog_manager.cc:4841] TS edf73cc1b3324b8caf421aeae37902e6 (127.7.132.130:41517): tablet 2336ff2466ed4a7caac0e374e46df656 (table test-table [id=4e5fe14f70d14af3a33e402c63b18818]) successfully deleted
I20260812 03:48:01.951710 16162 ts_tablet_manager.cc:1918] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:01.951802 16162 log.cc:1196] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/2336ff2466ed4a7caac0e374e46df656
I20260812 03:48:01.952112 16162 ts_tablet_manager.cc:1939] T 2336ff2466ed4a7caac0e374e46df656 P aaa156388a4e474780b0b0734db309cc: Deleting consensus metadata
I20260812 03:48:01.952565 15363 catalog_manager.cc:4841] TS aaa156388a4e474780b0b0734db309cc (127.7.132.131:39323): tablet 2336ff2466ed4a7caac0e374e46df656 (table test-table [id=4e5fe14f70d14af3a33e402c63b18818]) successfully deleted
W20260812 03:48:01.954263 15363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:38521 (error 108)
I20260812 03:48:01.954793 16118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:01.954631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:38521 (local address 127.0.0.1:37602)
0812 03:48:01.954689 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:01.954767 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:48:01.954916 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 15691
W20260812 03:48:01.954934 15363 catalog_manager.cc:4584] TS cf72bc3e99654df4817ae33ae5d24c8b (127.7.132.129:38521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2336ff2466ed4a7caac0e374e46df656: Network error: Client connection negotiation failed: client connection to 127.7.132.129:38521: connect: Connection refused (error 111)
I20260812 03:48:01.962257 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 15900
I20260812 03:48:01.970852 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 15328
2026-08-12T03:48:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (710 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 03:48:01.995774 7698 mini_oidc.cc:152] Starting JWKS server
I20260812 03:48:01.996930 7698 webserver.cc:460] Webserver started at https://127.0.0.1:42983/ using document root <none> and password file <none>
I20260812 03:48:01.997041 7698 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:48:01.997218 7698 webserver.cc:460] Webserver started at http://127.0.0.1:34769/ using document root <none> and password file <none>
2026-08-12T03:48:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:01Z Disabled control of system clock
I20260812 03:48:02.005541 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:36427
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:39657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:36427
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42983/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/testchainca.cert with env {}
W20260812 03:48:02.089452 16178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:02.089663 16178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:02.089685 16178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:02.090816 16178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:02.090852 16178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:02.090865 16178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:02.090878 16178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:02.090890 16178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:48:02.090902 16178 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:48:02.090914 16178 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42983/jwks.json
I20260812 03:48:02.092430 16178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:36427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:36427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42983/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:02.092633 16178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:02.092808 16178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:02.097635 16190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:02.097762 16178 server_base.cc:1034] running on GCE node
W20260812 03:48:02.097887 16186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:02.097926 16187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:02.098207 16178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:02.098488 16178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:02.099646 16178 hybrid_clock.cc:648] HybridClock initialized: now 1786506482099628 us; error 32 us; skew 500 ppm
I20260812 03:48:02.100826 16178 webserver.cc:460] Webserver started at http://127.7.132.190:41449/ using document root <none> and password file <none>
I20260812 03:48:02.101068 16178 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:02.101140 16178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:02.101305 16178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:02.102421 16178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "e67d82414a7745a0bb46223035eb7e74"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "c3232d7d94d2be61fc9be362ff9162a0"
server_key_iv: "61ae35693cc16ba823b9268872ca5731"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.102840 16178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "e67d82414a7745a0bb46223035eb7e74"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "c3232d7d94d2be61fc9be362ff9162a0"
server_key_iv: "61ae35693cc16ba823b9268872ca5731"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.104146 16178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:02.104863 16202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:02.105041 16178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:02.105144 16178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "e67d82414a7745a0bb46223035eb7e74"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "c3232d7d94d2be61fc9be362ff9162a0"
server_key_iv: "61ae35693cc16ba823b9268872ca5731"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.105240 16178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:02.139767 16178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:02.140123 16178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:02.140298 16178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:02.144537 16178 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:36427
I20260812 03:48:02.144577 16282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:36427 every 8 connection(s)
I20260812 03:48:02.144940 16178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:02.145972 16284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:02.147152 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 16178
I20260812 03:48:02.147264 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:02.147504 7698 external_mini_cluster.cc:1428] Setting key e9090757bef8944bd6b1c948d5bb488a
I20260812 03:48:02.149029 16284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74: Bootstrap starting.
I20260812 03:48:02.149667 16284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:02.149968 16284 log.cc:824] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:02.150803 16284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74: No bootstrap required, opened a new log
I20260812 03:48:02.153191 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:02.148018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.7.132.190:36427)
0812 03:48:02.148227 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:02.148240 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:02.148264 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:48:02.148762 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:02.148768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:02.148837 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:02.148947 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:02.149223 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.150198 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.151331 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.151710 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.151805 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.152503 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:02.152506 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:02.152507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:02.152670 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:02.152701 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:02.152711 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:02.152773 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:02.152885 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 03:48:02.152967 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260812 03:48:02.153146 16284 raft_consensus.cc:348] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d82414a7745a0bb46223035eb7e74" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36427 } }
I20260812 03:48:02.153297 16284 raft_consensus.cc:374] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:02.153332 16284 raft_consensus.cc:729] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e67d82414a7745a0bb46223035eb7e74, State: Initialized, Role: FOLLOWER
I20260812 03:48:02.153450 16284 consensus_queue.cc:260] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d82414a7745a0bb46223035eb7e74" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36427 } }
I20260812 03:48:02.153522 16284 raft_consensus.cc:388] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:02.153544 16284 raft_consensus.cc:482] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:02.153612 16284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:02.154313 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:48:02.154321 16284 raft_consensus.cc:504] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d82414a7745a0bb46223035eb7e74" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36427 } }
I20260812 03:48:02.154438 16284 leader_election.cc:302] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e67d82414a7745a0bb46223035eb7e74; no voters:
I20260812 03:48:02.154597 16284 leader_election.cc:288] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:02.154628 16295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:02.154768 16295 raft_consensus.cc:686] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [term 1 LEADER]: Becoming Leader. State: Replica: e67d82414a7745a0bb46223035eb7e74, State: Running, Role: LEADER
I20260812 03:48:02.154881 16284 sys_catalog.cc:564] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:02.154852 16295 consensus_queue.cc:237] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d82414a7745a0bb46223035eb7e74" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36427 } }
I20260812 03:48:02.155170 16301 sys_catalog.cc:455] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e67d82414a7745a0bb46223035eb7e74. Latest consensus state: current_term: 1 leader_uuid: "e67d82414a7745a0bb46223035eb7e74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d82414a7745a0bb46223035eb7e74" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36427 } } }
I20260812 03:48:02.155268 16301 sys_catalog.cc:458] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:02.155202 16296 sys_catalog.cc:455] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e67d82414a7745a0bb46223035eb7e74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e67d82414a7745a0bb46223035eb7e74" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36427 } } }
I20260812 03:48:02.155403 16296 sys_catalog.cc:458] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:02.155486 16305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:02.156049 16305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:02.157507 16305 catalog_manager.cc:1347] Generated new cluster ID: f524eedfd08d4a47b80b395da52fbfb2
I20260812 03:48:02.157560 16305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:02.165426 16305 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:02.165874 16305 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:02.173382 16305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e67d82414a7745a0bb46223035eb7e74: Generated new TSK 0
I20260812 03:48:02.173578 16305 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:48:02.202205 16342 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 03:48:02.202359 16342 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:48:02.202589 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:02.186102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42142 (local address 127.7.132.190:36427)
0812 03:48:02.186163 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:02.186166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:02.186298 (+ 132us) server_negotiation.cc:407] Connection header received
0812 03:48:02.186394 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:02.186394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:02.186404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:02.186459 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:48:02.186624 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.187147 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.188266 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.188568 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.188608 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.188946 (+ 338us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:48:02.202334 (+ 13388us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:48:02.202499 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42142: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 03:48:02.202670 16290 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42142: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 03:48:02.203168 16341 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:02.186039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:36427 (local address 127.0.0.1:42142)
0812 03:48:02.186227 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:02.186239 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:02.186308 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:02.186487 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:02.186489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:02.186490 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:48:02.186561 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:02.186563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.187218 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.187219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:02.188163 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:02.188166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.188674 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.188674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:02.188746 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.188766 (+ 20us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:48:02.202528 (+ 13762us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:48:02.202650 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.132.190:36427: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.132.190:36427: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42983/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 03:48:02.203598 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 16178
2026-08-12T03:48:02Z chronyd exiting
I20260812 03:48:02.259498 7698 test_util.cc:175] -----------------------------------------------
I20260812 03:48:02.259593 7698 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786506457253237-7698-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (264 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 03:48:02.260320 7698 mini_oidc.cc:152] Starting JWKS server
I20260812 03:48:02.261006 7698 webserver.cc:460] Webserver started at https://127.0.0.1:35375/ using document root <none> and password file <none>
I20260812 03:48:02.261077 7698 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:48:02.261260 7698 webserver.cc:460] Webserver started at http://127.0.0.1:42345/ using document root <none> and password file <none>
2026-08-12T03:48:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:02Z Disabled control of system clock
I20260812 03:48:02.269750 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:36539
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:36539
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35375/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/testchainca.cert with env {}
W20260812 03:48:02.348491 16354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:02.348664 16354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:02.348685 16354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:02.349821 16354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:02.349860 16354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:02.349872 16354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:02.349885 16354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:02.349936 16354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:48:02.349954 16354 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:48:02.349979 16354 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35375/jwks.json
I20260812 03:48:02.351511 16354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:36539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:36539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35375/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:02.351786 16354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:02.351994 16354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:02.356739 16365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:02.356777 16364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:02.356765 16368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:02.356954 16354 server_base.cc:1034] running on GCE node
I20260812 03:48:02.357147 16354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:02.357379 16354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:02.358512 16354 hybrid_clock.cc:648] HybridClock initialized: now 1786506482358500 us; error 31 us; skew 500 ppm
I20260812 03:48:02.359750 16354 webserver.cc:460] Webserver started at http://127.7.132.190:34949/ using document root <none> and password file <none>
I20260812 03:48:02.359928 16354 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:02.359968 16354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:02.360050 16354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:02.360980 16354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "552c4f7da4cc467bb9085773565af583"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "6ad52b168f02396002bd6f2d1c3f41ce"
server_key_iv: "0190c23a7ea6fa2193423994c6a29489"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.361275 16354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "552c4f7da4cc467bb9085773565af583"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "6ad52b168f02396002bd6f2d1c3f41ce"
server_key_iv: "0190c23a7ea6fa2193423994c6a29489"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.362445 16354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:02.363171 16378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:02.363337 16354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:02.363425 16354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "552c4f7da4cc467bb9085773565af583"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "6ad52b168f02396002bd6f2d1c3f41ce"
server_key_iv: "0190c23a7ea6fa2193423994c6a29489"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.363497 16354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:02.388269 16354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:02.388559 16354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:02.388701 16354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:02.392490 16354 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:36539
I20260812 03:48:02.392556 16449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:36539 every 8 connection(s)
I20260812 03:48:02.392858 16354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:02.393764 16450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:02.394129 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 16354
I20260812 03:48:02.394228 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:02.394418 7698 external_mini_cluster.cc:1428] Setting key 40ff013ca528134a2897450736156be4
I20260812 03:48:02.396524 16450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583: Bootstrap starting.
I20260812 03:48:02.397207 16450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:02.397511 16450 log.cc:824] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:02.398195 16450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583: No bootstrap required, opened a new log
I20260812 03:48:02.399755 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:02.394925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34142 (local address 127.7.132.190:36539)
0812 03:48:02.395103 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:02.395117 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:02.395140 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:48:02.395540 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:02.395545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:02.395598 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:02.395709 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:02.395971 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.396841 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.397931 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.398356 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.398453 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.399108 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:02.399111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:02.399111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:02.399266 (+ 155us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:02.399296 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:02.399317 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:02.399365 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:02.399462 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 03:48:02.399513 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260812 03:48:02.400738 16450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552c4f7da4cc467bb9085773565af583" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36539 } }
I20260812 03:48:02.400861 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:48:02.400905 16450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:02.400954 16450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552c4f7da4cc467bb9085773565af583, State: Initialized, Role: FOLLOWER
I20260812 03:48:02.401068 16450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552c4f7da4cc467bb9085773565af583" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36539 } }
I20260812 03:48:02.401163 16450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:02.401207 16450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:02.401255 16450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:02.401952 16450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552c4f7da4cc467bb9085773565af583" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36539 } }
I20260812 03:48:02.402058 16450 leader_election.cc:302] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 552c4f7da4cc467bb9085773565af583; no voters:
I20260812 03:48:02.402240 16450 leader_election.cc:288] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:02.402283 16461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:02.402446 16461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [term 1 LEADER]: Becoming Leader. State: Replica: 552c4f7da4cc467bb9085773565af583, State: Running, Role: LEADER
I20260812 03:48:02.402488 16450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:02.402529 16461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552c4f7da4cc467bb9085773565af583" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36539 } }
I20260812 03:48:02.402901 16463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "552c4f7da4cc467bb9085773565af583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552c4f7da4cc467bb9085773565af583" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36539 } } }
I20260812 03:48:02.402899 16468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 552c4f7da4cc467bb9085773565af583. Latest consensus state: current_term: 1 leader_uuid: "552c4f7da4cc467bb9085773565af583" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552c4f7da4cc467bb9085773565af583" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 36539 } } }
I20260812 03:48:02.403012 16463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:02.403019 16468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:02.403690 16474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:02.404136 16474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:02.405661 16474 catalog_manager.cc:1347] Generated new cluster ID: e6d55b8d17fd4e90b867f7941fc0aa9a
I20260812 03:48:02.405715 16474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:02.426043 16474 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:02.426586 16474 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:02.437224 16474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 552c4f7da4cc467bb9085773565af583: Generated new TSK 0
I20260812 03:48:02.437397 16474 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:48:02.462345 16498 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 03:48:02.462435 16498 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:48:02.462771 16456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:02.443598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34154 (local address 127.7.132.190:36539)
0812 03:48:02.443682 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:02.443685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:02.443793 (+ 108us) server_negotiation.cc:407] Connection header received
0812 03:48:02.443876 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:02.443877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:02.443889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:02.443952 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:48:02.444081 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.444570 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.445307 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.445503 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.445602 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.445792 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:48:02.462527 (+ 16735us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35375/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:48:02.462678 (+ 151us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34154: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35375/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 03:48:02.462829 16456 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34154: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35375/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 03:48:02.462904 16496 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:02.443531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:36539 (local address 127.0.0.1:34154)
0812 03:48:02.443715 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:02.443725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:02.443810 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:02.443956 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:02.443957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:02.443959 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:48:02.444015 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:02.444017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.444674 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.444676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:02.445222 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:02.445224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.445607 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.445608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:02.445681 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.445692 (+ 11us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:48:02.462640 (+ 16948us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35375/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:48:02.462784 (+ 144us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.132.190:36539: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35375/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":120,"threads_started":1}
I20260812 03:48:02.463181 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 16354
2026-08-12T03:48:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 03:48:02.508219 7698 mini_oidc.cc:152] Starting JWKS server
I20260812 03:48:02.508931 7698 webserver.cc:460] Webserver started at https://127.0.0.1:36003/ using document root <none> and password file <none>
I20260812 03:48:02.508973 7698 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:48:02.509132 7698 webserver.cc:460] Webserver started at http://127.0.0.1:40533/ using document root <none> and password file <none>
2026-08-12T03:48:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:02Z Disabled control of system clock
I20260812 03:48:02.517411 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37165
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:46173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:37165
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36003/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:48:02.595618 16509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:02.595816 16509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:02.595839 16509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:02.596983 16509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:02.597023 16509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:02.597035 16509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:02.597047 16509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:02.597059 16509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:48:02.597069 16509 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:48:02.597080 16509 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36003/jwks.json
I20260812 03:48:02.598327 16509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:37165
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36003/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:02.598541 16509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:02.598869 16509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:02.603744 16527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:02.603729 16524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:02.603710 16519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:02.603987 16509 server_base.cc:1034] running on GCE node
I20260812 03:48:02.604233 16509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:02.604519 16509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:02.605702 16509 hybrid_clock.cc:648] HybridClock initialized: now 1786506482605677 us; error 37 us; skew 500 ppm
I20260812 03:48:02.606953 16509 webserver.cc:460] Webserver started at http://127.7.132.190:45891/ using document root <none> and password file <none>
I20260812 03:48:02.607184 16509 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:02.607251 16509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:02.607411 16509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:02.608453 16509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "9535d89b935043e1b5f2fb3da3e5fa2e"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "55e2c10462f9c09bb3a2aa56c00e87bc"
server_key_iv: "dddeec7ecad3b0a9e3f225ee52ee5882"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.608850 16509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "9535d89b935043e1b5f2fb3da3e5fa2e"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "55e2c10462f9c09bb3a2aa56c00e87bc"
server_key_iv: "dddeec7ecad3b0a9e3f225ee52ee5882"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.610401 16509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:02.611244 16538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:02.611426 16509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:02.611505 16509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "9535d89b935043e1b5f2fb3da3e5fa2e"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "55e2c10462f9c09bb3a2aa56c00e87bc"
server_key_iv: "dddeec7ecad3b0a9e3f225ee52ee5882"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.611562 16509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:02.622197 16509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:02.622437 16509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:02.622529 16509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:02.626389 16509 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:37165
I20260812 03:48:02.626423 16625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:37165 every 8 connection(s)
I20260812 03:48:02.626857 16509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:02.627606 16626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:02.630448 16626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e: Bootstrap starting.
I20260812 03:48:02.631115 16626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:02.631409 16626 log.cc:824] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:02.631727 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 16509
I20260812 03:48:02.631847 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:02.632062 7698 external_mini_cluster.cc:1428] Setting key 7fc8eb2e48d3eab19988807cea24ad96
I20260812 03:48:02.632179 16626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e: No bootstrap required, opened a new log
I20260812 03:48:02.634397 16626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535d89b935043e1b5f2fb3da3e5fa2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37165 } }
I20260812 03:48:02.634526 16626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:02.634547 16626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9535d89b935043e1b5f2fb3da3e5fa2e, State: Initialized, Role: FOLLOWER
I20260812 03:48:02.634672 16626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535d89b935043e1b5f2fb3da3e5fa2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37165 } }
I20260812 03:48:02.634824 16626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:02.634869 16626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:02.634917 16626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:02.635439 16626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535d89b935043e1b5f2fb3da3e5fa2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37165 } }
I20260812 03:48:02.635556 16626 leader_election.cc:302] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9535d89b935043e1b5f2fb3da3e5fa2e; no voters:
I20260812 03:48:02.635744 16626 leader_election.cc:288] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:02.635766 16636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:02.635878 16636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [term 1 LEADER]: Becoming Leader. State: Replica: 9535d89b935043e1b5f2fb3da3e5fa2e, State: Running, Role: LEADER
I20260812 03:48:02.635959 16636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535d89b935043e1b5f2fb3da3e5fa2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37165 } }
I20260812 03:48:02.636011 16626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:02.636269 16638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9535d89b935043e1b5f2fb3da3e5fa2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535d89b935043e1b5f2fb3da3e5fa2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37165 } } }
I20260812 03:48:02.636518 16638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:02.636453 16639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9535d89b935043e1b5f2fb3da3e5fa2e. Latest consensus state: current_term: 1 leader_uuid: "9535d89b935043e1b5f2fb3da3e5fa2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9535d89b935043e1b5f2fb3da3e5fa2e" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37165 } } }
I20260812 03:48:02.636701 16639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:02.636746 16650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:02.637264 16650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:02.638012 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:02.632577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45218 (local address 127.7.132.190:37165)
0812 03:48:02.632765 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:02.632774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:02.632804 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:48:02.633369 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:02.633376 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:02.633430 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:02.633570 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:02.633800 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.634737 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.635533 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.636189 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.636342 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.637059 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:02.637063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:02.637064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:02.637272 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:02.637309 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:02.637323 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:02.637613 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:02.637643 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 03:48:02.637721 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260812 03:48:02.638965 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:48:02.639011 16650 catalog_manager.cc:1347] Generated new cluster ID: 15aff4fe34cf4eecbb67f36825816274
I20260812 03:48:02.639046 16650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:02.650432 16650 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:02.651137 16650 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:02.666203 16650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9535d89b935043e1b5f2fb3da3e5fa2e: Generated new TSK 0
I20260812 03:48:02.666374 16650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:48:02.680269 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:02.676760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45226 (local address 127.7.132.190:37165)
0812 03:48:02.676817 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:02.676819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:02.676886 (+ 67us) server_negotiation.cc:407] Connection header received
0812 03:48:02.676954 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:02.676955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:02.676967 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:02.677038 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:48:02.677146 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.677915 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.678609 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.678948 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.679018 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.679158 (+ 140us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:48:02.680062 (+ 904us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:48:02.680205 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45226: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 03:48:02.680321 16633 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45226: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 03:48:02.682197 16679 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:02.676655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:37165 (local address 127.0.0.1:45226)
0812 03:48:02.676829 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:02.676837 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:02.676886 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:02.677040 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:02.677040 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:02.677042 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:48:02.677094 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:02.677096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.677990 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.677991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:02.678535 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:02.678536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.679021 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.679022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:02.679066 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.679070 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:48:02.680182 (+ 1112us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:48:02.682100 (+ 1918us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.132.190:37165: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":1697,"thread_start_us":108,"threads_started":1}
I20260812 03:48:02.682469 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 16509
E20260812 03:48:02.697376 16680 webserver.cc:533] Webserver: sslize failed:
E20260812 03:48:02.697441 16680 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T03:48:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (219 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T03:48:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:02Z Disabled control of system clock
I20260812 03:48:02.735495 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38631
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38631
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/testchainca.cert with env {}
W20260812 03:48:02.813647 16685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:02.813825 16685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:02.813845 16685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:02.814953 16685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:02.814991 16685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:02.815003 16685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:02.815014 16685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:02.815065 16685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:48:02.815119 16685 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:48:02.815152 16685 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 03:48:02.816586 16685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38631
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:02.816797 16685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:02.816987 16685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:02.821575 16694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:02.821593 16691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:02.821581 16692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:02.821913 16685 server_base.cc:1034] running on GCE node
I20260812 03:48:02.822142 16685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:02.822402 16685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:02.823539 16685 hybrid_clock.cc:648] HybridClock initialized: now 1786506482823533 us; error 34 us; skew 500 ppm
I20260812 03:48:02.824630 16685 webserver.cc:460] Webserver started at http://127.7.132.190:39423/ using document root <none> and password file <none>
I20260812 03:48:02.824818 16685 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:02.824857 16685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:02.824941 16685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:02.825860 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "c7c0c78486604ae29dea433f2ffac9a5"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "57656e6683f5954316582fe3bad6a79c"
server_key_iv: "bfbe863eff407caef91a6436c10c1a3d"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.826154 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "c7c0c78486604ae29dea433f2ffac9a5"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "57656e6683f5954316582fe3bad6a79c"
server_key_iv: "bfbe863eff407caef91a6436c10c1a3d"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.827399 16685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:02.828056 16704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:02.828253 16685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:02.828377 16685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "c7c0c78486604ae29dea433f2ffac9a5"
format_stamp: "Formatted at 2026-08-12 03:48:02 on dist-test-slave-39l8"
server_key: "57656e6683f5954316582fe3bad6a79c"
server_key_iv: "bfbe863eff407caef91a6436c10c1a3d"
server_key_version: "encryptionkey@0"
I20260812 03:48:02.828449 16685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:02.853669 16685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:02.853973 16685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:02.854171 16685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:02.858637 16685 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38631
I20260812 03:48:02.858667 16794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38631 every 8 connection(s)
I20260812 03:48:02.859041 16685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:02.859676 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 16685
I20260812 03:48:02.859797 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:02.860030 7698 external_mini_cluster.cc:1428] Setting key 7d4f444ca9dfbf693c7205c990fc8db6
I20260812 03:48:02.860157 16796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:02.863353 16796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5: Bootstrap starting.
I20260812 03:48:02.863940 16796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:02.864238 16796 log.cc:824] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:02.864930 16796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5: No bootstrap required, opened a new log
I20260812 03:48:02.865648 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:02.860572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42692 (local address 127.7.132.190:38631)
0812 03:48:02.860766 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:02.860774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:02.860805 (+ 31us) server_negotiation.cc:407] Connection header received
0812 03:48:02.861341 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:02.861347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:02.861405 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:02.861514 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:02.861756 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.862589 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.863615 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.863995 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.864101 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.864955 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:02.864959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:02.864960 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:02.865157 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:02.865197 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:02.865210 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:02.865279 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:02.865369 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:48:02.865445 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260812 03:48:02.866700 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:48:02.867319 16796 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c0c78486604ae29dea433f2ffac9a5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38631 } }
I20260812 03:48:02.867484 16796 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:02.867528 16796 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7c0c78486604ae29dea433f2ffac9a5, State: Initialized, Role: FOLLOWER
I20260812 03:48:02.867641 16796 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c0c78486604ae29dea433f2ffac9a5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38631 } }
I20260812 03:48:02.867720 16796 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:02.867750 16796 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:02.867800 16796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:02.868320 16796 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c0c78486604ae29dea433f2ffac9a5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38631 } }
I20260812 03:48:02.868458 16796 leader_election.cc:302] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7c0c78486604ae29dea433f2ffac9a5; no voters:
I20260812 03:48:02.868631 16796 leader_election.cc:288] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:02.868700 16803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:02.868835 16803 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [term 1 LEADER]: Becoming Leader. State: Replica: c7c0c78486604ae29dea433f2ffac9a5, State: Running, Role: LEADER
I20260812 03:48:02.868881 16796 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:02.868975 16803 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c0c78486604ae29dea433f2ffac9a5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38631 } }
I20260812 03:48:02.869491 16805 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7c0c78486604ae29dea433f2ffac9a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c0c78486604ae29dea433f2ffac9a5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38631 } } }
I20260812 03:48:02.869493 16806 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7c0c78486604ae29dea433f2ffac9a5. Latest consensus state: current_term: 1 leader_uuid: "c7c0c78486604ae29dea433f2ffac9a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c0c78486604ae29dea433f2ffac9a5" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38631 } } }
I20260812 03:48:02.869591 16805 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:02.869606 16806 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:02.869818 16819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:02.870333 16819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:02.871801 16819 catalog_manager.cc:1347] Generated new cluster ID: 682005ad3861477ca00f6b78292edc60
I20260812 03:48:02.871905 16819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:02.891484 16819 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:02.891985 16819 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:02.898485 16819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7c0c78486604ae29dea433f2ffac9a5: Generated new TSK 0
I20260812 03:48:02.898669 16819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:48:03.055279 16801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:02.912794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42696 (local address 127.7.132.190:38631)
0812 03:48:02.912843 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:02.912845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:02.913153 (+ 308us) server_negotiation.cc:407] Connection header received
0812 03:48:02.913228 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:02.913229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:02.913243 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:02.913323 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:48:02.913503 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.914272 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.918760 (+ 4488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.918990 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.919050 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.919301 (+ 251us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:48:03.055046 (+135745us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:48:03.055214 (+ 168us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42696: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 03:48:03.055342 16801 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42696: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 03:48:03.055456 16837 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:02.912890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:38631 (local address 127.0.0.1:42696)
0812 03:48:02.913065 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:02.913083 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:02.913151 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:02.913352 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:02.913354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:02.913355 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:48:02.913424 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:02.913426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.914372 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.914373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:02.915321 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:02.915324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:02.919083 (+ 3759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:02.919084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:02.919161 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:02.919168 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:48:03.055197 (+136029us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:48:03.055322 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.132.190:38631: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260812 03:48:03.055788 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 16685
2026-08-12T03:48:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (357 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (89 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:48:03 dist-test-slave-39l8 krb5kdc[16856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:48:03 dist-test-slave-39l8 krb5kdc[16856](info): set up 2 sockets
Aug 12 03:48:03 dist-test-slave-39l8 krb5kdc[16856](info): commencing operation
krb5kdc: starting...
W20260812 03:48:04.441749 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:48:04 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:48:04.553406 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34965
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:34965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:04.630483 16874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:04.630702 16874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:04.630757 16874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:04.631873 16874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:04.631939 16874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:04.631961 16874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:04.631982 16874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:04.632010 16874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:04.633422 16874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:34965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:04.633674 16874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:04.633884 16874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:04.638540 16884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:04.638603 16885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:04.638604 16887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:04.638810 16874 server_base.cc:1034] running on GCE node
I20260812 03:48:04.639063 16874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:04.639338 16874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:04.640496 16874 hybrid_clock.cc:648] HybridClock initialized: now 1786506484640493 us; error 37 us; skew 500 ppm
Aug 12 03:48:04 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:04.646953 16874 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:48:04.647418 16874 webserver.cc:460] Webserver started at http://127.7.132.190:37443/ using document root <none> and password file <none>
I20260812 03:48:04.647650 16874 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:04.647717 16874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:04.647873 16874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:04.648900 16874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "1b33e7ecc7e44f069606fcd33f796adc"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "9abc61f0992620ae26c69044325314cc"
server_key_iv: "9c04c3bc7734b26649e0b139024bd9c5"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.649276 16874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "1b33e7ecc7e44f069606fcd33f796adc"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "9abc61f0992620ae26c69044325314cc"
server_key_iv: "9c04c3bc7734b26649e0b139024bd9c5"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.650590 16874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:04.651361 16897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:04.651533 16874 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:04.651643 16874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "1b33e7ecc7e44f069606fcd33f796adc"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "9abc61f0992620ae26c69044325314cc"
server_key_iv: "9c04c3bc7734b26649e0b139024bd9c5"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.651721 16874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:04.662487 16874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:04.664561 16874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:04.664705 16874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:04.668432 16874 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:34965
I20260812 03:48:04.668478 16978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:34965 every 8 connection(s)
I20260812 03:48:04.668813 16874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:04.669637 16980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:04.671840 16980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc: Bootstrap starting.
I20260812 03:48:04.672463 16980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:04.672745 16980 log.cc:824] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:04.673408 16980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc: No bootstrap required, opened a new log
I20260812 03:48:04.675033 16980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b33e7ecc7e44f069606fcd33f796adc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34965 } }
I20260812 03:48:04.675166 16980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:04.675189 16980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b33e7ecc7e44f069606fcd33f796adc, State: Initialized, Role: FOLLOWER
I20260812 03:48:04.675348 16980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b33e7ecc7e44f069606fcd33f796adc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34965 } }
I20260812 03:48:04.675426 16980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:04.675468 16980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:04.675539 16980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:04.676069 16980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b33e7ecc7e44f069606fcd33f796adc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34965 } }
I20260812 03:48:04.676152 16980 leader_election.cc:302] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b33e7ecc7e44f069606fcd33f796adc; no voters:
I20260812 03:48:04.676272 16980 leader_election.cc:288] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:04.676352 16986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:04.676509 16986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [term 1 LEADER]: Becoming Leader. State: Replica: 1b33e7ecc7e44f069606fcd33f796adc, State: Running, Role: LEADER
I20260812 03:48:04.676538 16980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:04.676599 16986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b33e7ecc7e44f069606fcd33f796adc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34965 } }
I20260812 03:48:04.677017 16988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b33e7ecc7e44f069606fcd33f796adc. Latest consensus state: current_term: 1 leader_uuid: "1b33e7ecc7e44f069606fcd33f796adc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b33e7ecc7e44f069606fcd33f796adc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34965 } } }
I20260812 03:48:04.677112 16988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:04.677136 16987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b33e7ecc7e44f069606fcd33f796adc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b33e7ecc7e44f069606fcd33f796adc" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34965 } } }
I20260812 03:48:04.677212 16987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:04.677345 16998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:04.677958 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 16874
I20260812 03:48:04.678064 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:04.678062 16998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:04.678282 7698 external_mini_cluster.cc:1428] Setting key b0964bdab30c0a840cecba6e18793ee6
I20260812 03:48:04.679657 16998 catalog_manager.cc:1347] Generated new cluster ID: f97864e4f2064175811bfcb9c74b5764
I20260812 03:48:04.679703 16998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:48:04 dist-test-slave-39l8 krb5kdc[16856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:04.692005 16998 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:04.692627 16998 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:04.693899 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:04.678835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56896 (local address 127.7.132.190:34965)
0812 03:48:04.678990 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:04.678999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:04.683009 (+ 4010us) server_negotiation.cc:407] Connection header received
0812 03:48:04.683130 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:04.683138 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:04.683191 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:04.683344 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:04.684051 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.685934 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.686950 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.687577 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.687685 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:04.690238 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:04.690242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:04.690242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:04.690253 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:04.692279 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:04.692699 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:04.692702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:04.692702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:04.692825 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:04.693049 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:04.693050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:04.693050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:04.693295 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:04.693434 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:04.693541 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 03:48:04.693683 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260812 03:48:04.696540 16998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b33e7ecc7e44f069606fcd33f796adc: Generated new TSK 0
I20260812 03:48:04.696683 16998 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:48:04.720729 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34965
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:04.801038 17021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:04.801275 17021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:04.801317 17021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:04.802462 17021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:04.802524 17021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:04.802565 17021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:04.804100 17021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:04.804344 17021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:04.804554 17021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:04.809525 17030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:04.809507 17033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:04.809540 17021 server_base.cc:1034] running on GCE node
W20260812 03:48:04.809510 17031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:04.809921 17021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:04.810149 17021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:04.811299 17021 hybrid_clock.cc:648] HybridClock initialized: now 1786506484811290 us; error 29 us; skew 500 ppm
Aug 12 03:48:04 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:04.813841 17021 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:48:04.814208 17021 webserver.cc:460] Webserver started at http://127.7.132.129:39821/ using document root <none> and password file <none>
I20260812 03:48:04.814368 17021 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:04.814405 17021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:04.814486 17021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:04.815567 17021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "8ae1a6980c1d4093922724b9bc0cfcd6"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "d8bc8c832ea29849e813100b661dcd93"
server_key_iv: "c4d705d3e29eef8a8e94a175b079e39c"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.815878 17021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "8ae1a6980c1d4093922724b9bc0cfcd6"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "d8bc8c832ea29849e813100b661dcd93"
server_key_iv: "c4d705d3e29eef8a8e94a175b079e39c"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.817066 17021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:48:04.817742 17042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:04.817891 17021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:04.817976 17021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "8ae1a6980c1d4093922724b9bc0cfcd6"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "d8bc8c832ea29849e813100b661dcd93"
server_key_iv: "c4d705d3e29eef8a8e94a175b079e39c"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.818028 17021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:04.824694 17021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:04.826386 17021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:04.826491 17021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:04.826658 17021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:04.826970 17021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:04.827004 17021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:04.827036 17021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:04.827049 17021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:04.832561 17021 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:42247
I20260812 03:48:04.832633 17202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:42247 every 8 connection(s)
I20260812 03:48:04.832919 17021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:48:04.835506 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 17021
I20260812 03:48:04.835623 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:04.835815 7698 external_mini_cluster.cc:1428] Setting key f296a6a90488b263c2393a214c37e7b9
Aug 12 03:48:04 dist-test-slave-39l8 krb5kdc[16856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:04.842964 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:04.833635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:57099 (local address 127.7.132.190:34965)
0812 03:48:04.833767 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:04.833770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:04.833915 (+ 145us) server_negotiation.cc:407] Connection header received
0812 03:48:04.834880 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:04.834881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:04.834895 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:04.834950 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:04.835543 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.836199 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.837233 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.837456 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.837529 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:04.839823 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:04.839824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:04.839824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:04.839832 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:04.841552 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:04.841942 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:04.841944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:04.841944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:04.842009 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:04.842258 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:04.842259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:04.842259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:04.842406 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:04.842484 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:04.842746 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 03:48:04.842842 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:48:04.842974 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:04.833588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34965 (local address 127.7.132.129:57099)
0812 03:48:04.833795 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:04.833838 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:04.834773 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:04.835012 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:04.835016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:04.835260 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:04.835445 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:04.835449 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.836297 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.836298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:04.837141 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:04.837144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.837550 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.837551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:04.837668 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:04.838194 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:04.838205 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:04.839731 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:04.841643 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:04.841650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:04.841653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:04.841867 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:04.842082 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:04.842082 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:04.842082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:04.842192 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:04.842476 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:04.842480 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:04.842581 (+ 101us) client_negotiation.cc:769] Sending connection context
0812 03:48:04.842658 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:48:04.842829 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260812 03:48:04.843489 17203 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34965
I20260812 03:48:04.843595 17203 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:04.843775 17203 heartbeater.cc:511] Master 127.7.132.190:34965 requested a full tablet report, sending...
I20260812 03:48:04.844312 16921 ts_manager.cc:194] Registered new tserver with Master: 8ae1a6980c1d4093922724b9bc0cfcd6 (127.7.132.129:42247)
I20260812 03:48:04.845227 16921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:57099
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:48:04.861851 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34965
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:04.941120 17209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:04.941318 17209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:04.941339 17209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:04.942471 17209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:04.942508 17209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:04.942579 17209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:04.943992 17209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:04.944235 17209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:04.944454 17209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:04.949215 17220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:04.949251 17218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:04.949391 17209 server_base.cc:1034] running on GCE node
W20260812 03:48:04.949214 17216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:04.949759 17209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:04.950016 17209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:04.951151 17209 hybrid_clock.cc:648] HybridClock initialized: now 1786506484951138 us; error 29 us; skew 500 ppm
Aug 12 03:48:04 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:04.953537 17209 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:48:04.953995 17209 webserver.cc:460] Webserver started at http://127.7.132.130:37441/ using document root <none> and password file <none>
I20260812 03:48:04.954209 17209 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:04.954277 17209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:04.954423 17209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:04.955425 17209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "0c84b486292142fa97c97185bf918949"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "e86da589a0cb6ef27072fe0aafd06c69"
server_key_iv: "eb77a50086a467a11d014399b2b94b47"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.955777 17209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "0c84b486292142fa97c97185bf918949"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "e86da589a0cb6ef27072fe0aafd06c69"
server_key_iv: "eb77a50086a467a11d014399b2b94b47"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.957002 17209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:48:04.957707 17230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:04.957875 17209 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:04.957980 17209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "0c84b486292142fa97c97185bf918949"
format_stamp: "Formatted at 2026-08-12 03:48:04 on dist-test-slave-39l8"
server_key: "e86da589a0cb6ef27072fe0aafd06c69"
server_key_iv: "eb77a50086a467a11d014399b2b94b47"
server_key_version: "encryptionkey@0"
I20260812 03:48:04.958051 17209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:04.975585 17209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:04.977703 17209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:04.977867 17209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:04.978094 17209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:04.978466 17209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:04.978528 17209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:04.978566 17209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:04.978593 17209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:04.984280 17209 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37501
I20260812 03:48:04.984333 17392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37501 every 8 connection(s)
I20260812 03:48:04.984691 17209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:04.986451 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 17209
I20260812 03:48:04.986542 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:48:04.986824 7698 external_mini_cluster.cc:1428] Setting key c2478fa38ae144d85a58d42085fa4643
Aug 12 03:48:04 dist-test-slave-39l8 krb5kdc[16856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:04.995105 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:04.985483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:52515 (local address 127.7.132.190:34965)
0812 03:48:04.985549 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:04.985551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:04.985732 (+ 181us) server_negotiation.cc:407] Connection header received
0812 03:48:04.986762 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:04.986763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:04.986777 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:04.986823 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:04.987595 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.988317 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.989307 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.989547 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.989602 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:04.992298 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:04.992299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:04.992299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:04.992306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:04.993663 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:04.994139 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:04.994140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:04.994140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:04.994188 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:04.994487 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:04.994488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:04.994488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:04.994584 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:04.994632 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:04.994980 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 03:48:04.995043 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:48:04.995378 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:04.985455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34965 (local address 127.7.132.130:52515)
0812 03:48:04.985610 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:04.985651 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:04.986610 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:04.986954 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:04.986958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:04.987282 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:04.987493 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:04.987499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.988411 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.988412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:04.989207 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:04.989211 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:04.989631 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:04.989632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:04.989783 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:04.990259 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:04.990270 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:04.992193 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:04.993752 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:04.993759 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:04.993762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:04.994065 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:04.994265 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:04.994265 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:04.994266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:04.994412 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:04.994650 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:04.994656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:04.994838 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 03:48:04.994965 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 03:48:04.995127 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260812 03:48:04.995931 17393 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34965
I20260812 03:48:04.996019 17393 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:04.996187 17393 heartbeater.cc:511] Master 127.7.132.190:34965 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
I20260812 03:48:04.996522 16921 ts_manager.cc:194] Registered new tserver with Master: 0c84b486292142fa97c97185bf918949 (127.7.132.130:37501)
I20260812 03:48:04.996894 16921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:52515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:48:05.011560 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34965
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:05.092594 17400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:05.092803 17400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:05.092825 17400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:05.093845 17400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:05.093875 17400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:05.093897 17400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:05.095221 17400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:05.095395 17400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:05.095554 17400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:05.100459 17413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:05.100448 17410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:05.100544 17400 server_base.cc:1034] running on GCE node
W20260812 03:48:05.100448 17411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:05.100862 17400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:05.101115 17400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:05.102284 17400 hybrid_clock.cc:648] HybridClock initialized: now 1786506485102279 us; error 37 us; skew 500 ppm
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:05.104851 17400 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:48:05.105315 17400 webserver.cc:460] Webserver started at http://127.7.132.131:42635/ using document root <none> and password file <none>
I20260812 03:48:05.105531 17400 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:05.105599 17400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:05.105756 17400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:05.106892 17400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "05f92bc871174d21b602493dc089901e"
format_stamp: "Formatted at 2026-08-12 03:48:05 on dist-test-slave-39l8"
server_key: "af1ddb8c177a71a40ffbd90deacf1d4e"
server_key_iv: "cd7e42030fc7315c69b7d8452c3b7979"
server_key_version: "encryptionkey@0"
I20260812 03:48:05.107259 17400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "05f92bc871174d21b602493dc089901e"
format_stamp: "Formatted at 2026-08-12 03:48:05 on dist-test-slave-39l8"
server_key: "af1ddb8c177a71a40ffbd90deacf1d4e"
server_key_iv: "cd7e42030fc7315c69b7d8452c3b7979"
server_key_version: "encryptionkey@0"
I20260812 03:48:05.108532 17400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:48:05.109254 17423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:05.109421 17400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:05.109525 17400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "05f92bc871174d21b602493dc089901e"
format_stamp: "Formatted at 2026-08-12 03:48:05 on dist-test-slave-39l8"
server_key: "af1ddb8c177a71a40ffbd90deacf1d4e"
server_key_iv: "cd7e42030fc7315c69b7d8452c3b7979"
server_key_version: "encryptionkey@0"
I20260812 03:48:05.109596 17400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:05.126189 17400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:05.128396 17400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:05.128552 17400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:05.128779 17400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:05.129065 17400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:05.129117 17400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:05.129150 17400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:05.129179 17400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:05.134711 17400 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:37281
I20260812 03:48:05.134773 17585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:37281 every 8 connection(s)
I20260812 03:48:05.135057 17400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:05.136741 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 17400
I20260812 03:48:05.136863 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:05.137049 7698 external_mini_cluster.cc:1428] Setting key 8537f1a63d505b8e25d1f327c0e53764
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:05.144109 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:05.135769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:60451 (local address 127.7.132.190:34965)
0812 03:48:05.135861 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:05.135863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:05.136018 (+ 155us) server_negotiation.cc:407] Connection header received
0812 03:48:05.137148 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:05.137149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:05.137163 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:05.137235 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:05.138033 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.138500 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.139475 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.139643 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.139694 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:05.141608 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:05.141610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:05.141610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:05.141616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:05.143064 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:05.143393 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:05.143394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:05.143394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:05.143442 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:05.143641 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:05.143641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:05.143641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:05.143730 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:05.143776 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:05.143987 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 03:48:05.144051 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:48:05.144236 17593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:05.135691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34965 (local address 127.7.132.131:60451)
0812 03:48:05.135889 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:05.135929 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:05.137019 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:05.137372 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:05.137376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:05.137741 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:05.137931 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:05.137936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.138573 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.138574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:05.139392 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:05.139395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.139715 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.139715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:05.139806 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:05.140182 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:05.140190 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:05.141525 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:05.143128 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:05.143133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:05.143135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:05.143328 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:05.143499 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:05.143500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:05.143500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:05.143589 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:05.143783 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:05.143787 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:05.143884 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:48:05.143958 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:48:05.144062 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260812 03:48:05.144697 17587 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34965
I20260812 03:48:05.144796 17587 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:05.144960 17587 heartbeater.cc:511] Master 127.7.132.190:34965 requested a full tablet report, sending...
I20260812 03:48:05.145318 16921 ts_manager.cc:194] Registered new tserver with Master: 05f92bc871174d21b602493dc089901e (127.7.132.131:37281)
I20260812 03:48:05.145658 16921 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:60451
I20260812 03:48:05.148806 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/test-admin.keytab.
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:05.163923 7698 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 03:48:05.165616 17600 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:05.164789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.7.132.129:42247)
0812 03:48:05.164964 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:05.164968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:05.164977 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:05.165000 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:05.165002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:05.165015 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:05.165079 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:05.165522 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48572: BlockingRecv error: recv got EOF from 127.0.0.1:48572 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
W20260812 03:48:05.165680 7698 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:05.166671 7698 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:05.167565 7698 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:48:05.174675 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:05.168078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.7.132.129:42247)
0812 03:48:05.168147 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:05.168149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:05.168176 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:48:05.168232 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:05.168233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:05.168242 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:05.168293 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:05.168641 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.169350 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.169983 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.170272 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.170324 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:05.172374 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:05.172381 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:05.172381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:05.172388 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:05.173801 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:05.174089 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:05.174091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:05.174092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:05.174142 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:05.174288 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:05.174288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:05.174288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:05.174390 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:05.174475 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:05.174541 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:48:05.174599 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 03:48:05.176556 17600 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:05.175960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.7.132.129:42247)
0812 03:48:05.175993 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:05.175995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:05.176087 (+ 92us) server_negotiation.cc:407] Connection header received
0812 03:48:05.176161 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:05.176162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:05.176173 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:05.176229 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:05.176501 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48584: BlockingRecv error: recv got EOF from 127.0.0.1:48584 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 03:48:05.176760 7698 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:05.177702 7698 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:05.178617 7698 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:48:05.184873 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:05.179147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.7.132.129:42247)
0812 03:48:05.179208 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:05.179210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:05.179257 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:48:05.179307 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:05.179308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:05.179315 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:05.179359 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:05.179670 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.180106 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.180696 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.180869 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.180915 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:05.182852 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:05.182853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:05.182853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:05.182859 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:05.183982 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:05.184271 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:05.184272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:05.184272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:05.184319 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:05.184477 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:05.184477 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:05.184477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:05.184555 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:05.184622 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:05.184703 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 03:48:05.184770 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 03:48:05.186256 17600 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:05.185799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48596 (local address 127.7.132.129:42247)
0812 03:48:05.185835 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:05.185837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:05.185883 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:48:05.185935 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:05.185936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:05.185942 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:05.185972 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:05.186217 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48596: BlockingRecv error: recv got EOF from 127.0.0.1:48596 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 03:48:05.186323 7698 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786506457253237-7698-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:05.187269 7698 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:05.188063 7698 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[16856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:48:05.194777 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:05.188522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.7.132.129:42247)
0812 03:48:05.188580 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:05.188582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:05.188646 (+ 64us) server_negotiation.cc:407] Connection header received
0812 03:48:05.188731 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:05.188732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:05.188757 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:05.188817 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:05.189160 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.189596 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.190283 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:05.190484 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:05.190557 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:05.192421 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:05.192422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:05.192423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:05.192430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:05.193699 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:05.194059 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:05.194060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:05.194060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:05.194116 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:05.194314 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:05.194315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:05.194315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:05.194407 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:05.194493 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:05.194599 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 03:48:05.194666 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:48:05.195315 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 17021
I20260812 03:48:05.212049 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 17209
I20260812 03:48:05.224538 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 17400
I20260812 03:48:05.233502 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 16874
2026-08-12T03:48:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1991 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[17629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[17629](info): set up 2 sockets
Aug 12 03:48:05 dist-test-slave-39l8 krb5kdc[17629](info): commencing operation
krb5kdc: starting...
W20260812 03:48:06.441206 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:48:06 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:48:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.190.keytab.
Entry for principal oryx/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.190.keytab.
I20260812 03:48:06.551403 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40737
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.190.keytab
--principal=oryx/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:06.627511 17648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:06.627749 17648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:06.627786 17648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:06.628861 17648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:06.628923 17648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:06.628945 17648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:06.628971 17648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:06.629000 17648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:06.630313 17648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.190.keytab
--principal=oryx/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:06.630555 17648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:06.630775 17648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:06.635632 17656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:06.635632 17658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:06.635732 17648 server_base.cc:1034] running on GCE node
W20260812 03:48:06.635739 17655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:06.636101 17648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:06.636426 17648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:06.637594 17648 hybrid_clock.cc:648] HybridClock initialized: now 1786506486637566 us; error 40 us; skew 500 ppm
Aug 12 03:48:06 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:06.639953 17648 init.cc:377] Logged in from keytab as oryx/127.7.132.190@KRBTEST.COM (short username oryx)
I20260812 03:48:06.640357 17648 webserver.cc:460] Webserver started at http://127.7.132.190:42175/ using document root <none> and password file <none>
I20260812 03:48:06.640566 17648 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:06.640631 17648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:06.640781 17648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:06.641984 17648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "d6ff6e4c5d005ba39884a561e0a511db"
server_key_iv: "69a052d78b31b65d9b2fc7a666fc34f2"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.642409 17648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "d6ff6e4c5d005ba39884a561e0a511db"
server_key_iv: "69a052d78b31b65d9b2fc7a666fc34f2"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.643738 17648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:06.644528 17670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.644701 17648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:48:06.644800 17648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "d6ff6e4c5d005ba39884a561e0a511db"
server_key_iv: "69a052d78b31b65d9b2fc7a666fc34f2"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.644870 17648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:06.654485 17648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:06.656247 17648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:06.656397 17648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:06.660043 17648 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:40737
I20260812 03:48:06.660073 17749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:40737 every 8 connection(s)
I20260812 03:48:06.660382 17648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:06.661267 17751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:06.663410 17751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: Bootstrap starting.
I20260812 03:48:06.663902 17751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:06.664129 17751 log.cc:824] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:06.664692 17751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: No bootstrap required, opened a new log
I20260812 03:48:06.665766 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 17648
I20260812 03:48:06.665886 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:06.666055 7698 external_mini_cluster.cc:1428] Setting key fcd54466772a7189b2ae8f4bca8f3bf1
I20260812 03:48:06.666200 17751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } }
I20260812 03:48:06.666359 17751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:06.666390 17751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2be07f7ce6c84cf286e210ee9e2ac3c1, State: Initialized, Role: FOLLOWER
I20260812 03:48:06.666510 17751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } }
I20260812 03:48:06.666586 17751 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:06.666633 17751 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:06.666728 17751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:06.667429 17751 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } }
I20260812 03:48:06.667531 17751 leader_election.cc:302] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2be07f7ce6c84cf286e210ee9e2ac3c1; no voters:
I20260812 03:48:06.667671 17751 leader_election.cc:288] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:06.667693 17757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:06.667800 17757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 1 LEADER]: Becoming Leader. State: Replica: 2be07f7ce6c84cf286e210ee9e2ac3c1, State: Running, Role: LEADER
I20260812 03:48:06.667902 17751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:06.667932 17757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } }
I20260812 03:48:06.668272 17757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be07f7ce6c84cf286e210ee9e2ac3c1. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } }
I20260812 03:48:06.668519 17757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:06.668442 17761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } }
I20260812 03:48:06.668942 17761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:06.669087 17770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:06.669586 17770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:06.671227 17770 catalog_manager.cc:1347] Generated new cluster ID: 00a4a174bbbb4bd0a09cf3a8eab9f15c
I20260812 03:48:06.671283 17770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:48:06 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:06.680173 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:06.666523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.7.132.190:40737)
0812 03:48:06.666686 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:06.666698 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:06.666915 (+ 217us) server_negotiation.cc:407] Connection header received
0812 03:48:06.667297 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:06.667306 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:06.667360 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:06.667495 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:06.668258 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:06.669544 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:06.670331 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:06.670807 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:06.670918 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:06.676879 (+ 5961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:06.676882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:06.676883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:06.676893 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:06.678646 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:06.679034 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:06.679037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:06.679038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:06.679134 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:06.679310 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:06.679311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:06.679311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:06.679554 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:06.679648 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:06.679845 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 03:48:06.679961 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260812 03:48:06.691666 17770 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:06.692366 17770 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:06.701100 17770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: Generated new TSK 0
I20260812 03:48:06.701279 17770 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
I20260812 03:48:06.732698 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40737
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:06.811116 17789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:06.811318 17789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:06.811342 17789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:06.812469 17789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:06.812507 17789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:06.812556 17789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:06.813885 17789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:06.814096 17789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:06.814314 17789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:06.819046 17797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:06.819135 17789 server_base.cc:1034] running on GCE node
W20260812 03:48:06.819056 17800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:06.819056 17798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:06.819437 17789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:06.819636 17789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:06.820775 17789 hybrid_clock.cc:648] HybridClock initialized: now 1786506486820755 us; error 45 us; skew 500 ppm
Aug 12 03:48:06 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:06.823366 17789 init.cc:377] Logged in from keytab as oryx/127.7.132.129@KRBTEST.COM (short username oryx)
I20260812 03:48:06.823834 17789 webserver.cc:460] Webserver started at http://127.7.132.129:37857/ using document root <none> and password file <none>
I20260812 03:48:06.824044 17789 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:06.824085 17789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:06.824168 17789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:06.825070 17789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "ee97fa4ed1794eb09fdba3945f17ac97"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "b78531ef7e8b00e889219fd2bc8b7fd4"
server_key_iv: "a75442456b464168e5961841688e7173"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.825359 17789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "ee97fa4ed1794eb09fdba3945f17ac97"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "b78531ef7e8b00e889219fd2bc8b7fd4"
server_key_iv: "a75442456b464168e5961841688e7173"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.826565 17789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:48:06.827297 17811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.827461 17789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.827589 17789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "ee97fa4ed1794eb09fdba3945f17ac97"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "b78531ef7e8b00e889219fd2bc8b7fd4"
server_key_iv: "a75442456b464168e5961841688e7173"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.827672 17789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:06.840430 17789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:06.842631 17789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:06.842824 17789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:06.843063 17789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:06.843357 17789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:06.843406 17789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.843438 17789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:06.843482 17789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.849273 17789 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:39265
I20260812 03:48:06.849318 17976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:39265 every 8 connection(s)
I20260812 03:48:06.849582 17789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:48:06 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:06.857352 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 17789
I20260812 03:48:06.857468 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:06.857679 7698 external_mini_cluster.cc:1428] Setting key 9daf1bc554a12ac2a30bb5f896a155fe
I20260812 03:48:06.857841 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:06.850174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:35977 (local address 127.7.132.190:40737)
0812 03:48:06.850255 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:06.850257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:06.850434 (+ 177us) server_negotiation.cc:407] Connection header received
0812 03:48:06.850926 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:06.850927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:06.850940 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:06.850994 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:06.851837 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:06.852359 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:06.853243 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:06.853422 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:06.853468 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:06.855440 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:06.855441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:06.855441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:06.855460 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:06.856557 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:06.856895 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:06.856896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:06.856896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:06.856944 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:06.857173 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:06.857173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:06.857173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:06.857267 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:06.857318 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:06.857642 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 03:48:06.857708 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:48:06.857981 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:06.850156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:35977)
0812 03:48:06.850324 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:06.850355 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:06.850836 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:06.851042 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:06.851045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:06.851585 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:06.851757 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:06.851761 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:06.852435 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:06.852436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:06.853158 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:06.853159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:06.853489 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:06.853490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:06.853576 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:06.854028 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:06.854043 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:06.855359 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:06.856642 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:06.856649 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:06.856650 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:06.856836 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:06.857010 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:06.857010 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:06.857011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:06.857114 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:06.857340 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:06.857347 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:06.857507 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 03:48:06.857625 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 03:48:06.857783 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260812 03:48:06.858443 17977 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40737
I20260812 03:48:06.858533 17977 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:06.858758 17977 heartbeater.cc:511] Master 127.7.132.190:40737 requested a full tablet report, sending...
I20260812 03:48:06.859309 17703 ts_manager.cc:194] Registered new tserver with Master: ee97fa4ed1794eb09fdba3945f17ac97 (127.7.132.129:39265)
I20260812 03:48:06.860152 17703 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:35977
WARNING: no policy specified for oryx/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
I20260812 03:48:06.880859 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40737
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:06.958442 17982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:06.958629 17982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:06.958657 17982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:06.959776 17982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:06.959815 17982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:06.959841 17982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:06.961206 17982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:06.961390 17982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:06.961548 17982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:06.966307 17987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:06.966307 17990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:06.966466 17982 server_base.cc:1034] running on GCE node
W20260812 03:48:06.966315 17988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:06.966818 17982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:06.967097 17982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:06.968258 17982 hybrid_clock.cc:648] HybridClock initialized: now 1786506486968245 us; error 43 us; skew 500 ppm
Aug 12 03:48:06 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:06.970846 17982 init.cc:377] Logged in from keytab as oryx/127.7.132.130@KRBTEST.COM (short username oryx)
I20260812 03:48:06.971312 17982 webserver.cc:460] Webserver started at http://127.7.132.130:39853/ using document root <none> and password file <none>
I20260812 03:48:06.971536 17982 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:06.971580 17982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:06.971737 17982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:06.972757 17982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "18837fec8c8743aab7fe31912772198a"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "36d330fd0d0afe3ccfbc3b1af04bfb8e"
server_key_iv: "089c4edcb9d6e7652530772e04ad8a7c"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.973126 17982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "18837fec8c8743aab7fe31912772198a"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "36d330fd0d0afe3ccfbc3b1af04bfb8e"
server_key_iv: "089c4edcb9d6e7652530772e04ad8a7c"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.974375 17982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:48:06.975127 18001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.975293 17982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.975394 17982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "18837fec8c8743aab7fe31912772198a"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "36d330fd0d0afe3ccfbc3b1af04bfb8e"
server_key_iv: "089c4edcb9d6e7652530772e04ad8a7c"
server_key_version: "encryptionkey@0"
I20260812 03:48:06.975466 17982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:06.990834 17982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:06.993053 17982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:06.993227 17982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:06.993423 17982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:06.993695 17982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:06.993749 17982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.993785 17982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:06.993810 17982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:06.999188 17982 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:45091
I20260812 03:48:06.999265 18193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:45091 every 8 connection(s)
I20260812 03:48:06.999563 17982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:07.005276 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 17982
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:07.005396 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:48:07.005617 7698 external_mini_cluster.cc:1428] Setting key 1cf91ad72720d416e5961130da61d1a4
I20260812 03:48:07.008934 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.000255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.130:36727)
0812 03:48:07.000412 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.000451 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.000903 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.001144 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.001148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.001700 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.001894 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.001898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.002548 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.002549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.003334 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.003336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.003683 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.003683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.003784 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.004259 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.004267 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.005849 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.007535 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.007541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.007543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.007841 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.008070 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.008071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.008071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.008200 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.008480 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.008485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.008585 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.008658 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.008786 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260812 03:48:07.008941 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.000296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:36727 (local address 127.7.132.190:40737)
0812 03:48:07.000370 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.000372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.000541 (+ 169us) server_negotiation.cc:407] Connection header received
0812 03:48:07.000987 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.000988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.000998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.001053 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.001973 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.002470 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.003408 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.003600 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.003654 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.005949 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.005951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.005951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.005958 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.007459 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.007915 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.007916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.007916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.007993 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.008275 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.008276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.008276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.008414 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.008484 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.008723 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.008822 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:48:07.009433 18194 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40737
I20260812 03:48:07.009542 18194 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:07.009691 18194 heartbeater.cc:511] Master 127.7.132.190:40737 requested a full tablet report, sending...
I20260812 03:48:07.010030 17703 ts_manager.cc:194] Registered new tserver with Master: 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091)
I20260812 03:48:07.010392 17703 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:36727
WARNING: no policy specified for oryx/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
I20260812 03:48:07.030572 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40737
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:07.111335 18202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:07.111568 18202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:07.111605 18202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:07.112736 18202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:07.112807 18202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:07.112862 18202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:07.114261 18202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:40737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:07.114506 18202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:07.114734 18202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:48:07.119460 18202 server_base.cc:1034] running on GCE node
W20260812 03:48:07.119381 18210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.119391 18207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.119380 18208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:07.119839 18202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:07.120045 18202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:07.121201 18202 hybrid_clock.cc:648] HybridClock initialized: now 1786506487121183 us; error 35 us; skew 500 ppm
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:07.123618 18202 init.cc:377] Logged in from keytab as oryx/127.7.132.131@KRBTEST.COM (short username oryx)
I20260812 03:48:07.124058 18202 webserver.cc:460] Webserver started at http://127.7.132.131:38731/ using document root <none> and password file <none>
I20260812 03:48:07.124276 18202 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:07.124343 18202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:07.124491 18202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:07.125463 18202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "038eef5f4c8545f59b60655118824e7e"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "a9045df5285fb598f751be607d18608d"
server_key_iv: "0842762f595ba5317e6c86f0ecd8f1e0"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.125823 18202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "038eef5f4c8545f59b60655118824e7e"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "a9045df5285fb598f751be607d18608d"
server_key_iv: "0842762f595ba5317e6c86f0ecd8f1e0"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.127203 18202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:07.127947 18217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.128129 18202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:07.128242 18202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "038eef5f4c8545f59b60655118824e7e"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "a9045df5285fb598f751be607d18608d"
server_key_iv: "0842762f595ba5317e6c86f0ecd8f1e0"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.128314 18202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:07.144929 18202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:07.146600 18202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:07.146821 18202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:07.147050 18202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:07.147343 18202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:07.147403 18202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.147462 18202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:07.147486 18202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.153383 18202 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:44111
I20260812 03:48:07.153431 18398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:44111 every 8 connection(s)
I20260812 03:48:07.153704 18202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:07.155238 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 18202
I20260812 03:48:07.155344 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:07.155507 7698 external_mini_cluster.cc:1428] Setting key 832e77df02759fb2dd7b944a57324aa7
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:07.162549 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.154338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:47903 (local address 127.7.132.190:40737)
0812 03:48:07.154400 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.154402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.154590 (+ 188us) server_negotiation.cc:407] Connection header received
0812 03:48:07.155115 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.155116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.155129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.155211 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.156191 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.156688 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.157726 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.157927 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.158016 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.160001 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.160003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.160003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.160023 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.161126 (+ 1103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.161570 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.161571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.161572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.161633 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.161906 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.161907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.161908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.162022 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.162098 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.162365 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.162459 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:48:07.162743 18401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.154301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:47903)
0812 03:48:07.154478 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.154516 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.155020 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.155261 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.155264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.155896 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.156064 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.156068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.156810 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.156812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.157612 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.157614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.158024 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.158025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.158146 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.158532 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.158539 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.159903 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.161254 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.161261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.161263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.161479 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.161722 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.161723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.161723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.161829 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.162089 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.162096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.162225 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.162328 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.162484 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260812 03:48:07.163195 18399 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40737
I20260812 03:48:07.163291 18399 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:07.163453 18399 heartbeater.cc:511] Master 127.7.132.190:40737 requested a full tablet report, sending...
I20260812 03:48:07.163800 17703 ts_manager.cc:194] Registered new tserver with Master: 038eef5f4c8545f59b60655118824e7e (127.7.132.131:44111)
I20260812 03:48:07.164120 17703 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:47903
I20260812 03:48:07.167148 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.132.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.189.keytab.
Entry for principal oryx/127.7.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.189.keytab.
I20260812 03:48:07.191306 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:39453
--webserver_interface=127.7.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.189.keytab
--principal=oryx/127.7.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40737,127.7.132.189:39453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:07.273278 18405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:07.273495 18405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:07.273533 18405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:07.274653 18405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:07.274712 18405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:07.274752 18405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:07.274781 18405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:07.274803 18405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:07.276247 18405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40737,127.7.132.189:39453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.189.keytab
--principal=oryx/127.7.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:39453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
W20260812 03:48:07.276403 18405 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.132.190:40737,127.7.132.189:39453'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 03:48:07.276541 18405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:07.276751 18405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:07.281543 18418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.281570 18415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.281561 18413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:07.281778 18405 server_base.cc:1034] running on GCE node
I20260812 03:48:07.282003 18405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:07.282290 18405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:07.283427 18405 hybrid_clock.cc:648] HybridClock initialized: now 1786506487283426 us; error 39 us; skew 500 ppm
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:07.285795 18405 init.cc:377] Logged in from keytab as oryx/127.7.132.189@KRBTEST.COM (short username oryx)
I20260812 03:48:07.286288 18405 webserver.cc:460] Webserver started at http://127.7.132.189:33805/ using document root <none> and password file <none>
I20260812 03:48:07.286553 18405 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:07.286621 18405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:07.286793 18405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:07.287868 18405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data/instance:
uuid: "d528fb1a1c3f46229b4533c6386ed33b"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "482f2dea45f5d4deb6aacf591f5b3d1d"
server_key_iv: "e58da952bb09efd0a32c0483bf27ff81"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.288223 18405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal/instance:
uuid: "d528fb1a1c3f46229b4533c6386ed33b"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "482f2dea45f5d4deb6aacf591f5b3d1d"
server_key_iv: "e58da952bb09efd0a32c0483bf27ff81"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.289541 18405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:07.290266 18427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.290436 18405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:07.290541 18405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
uuid: "d528fb1a1c3f46229b4533c6386ed33b"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "482f2dea45f5d4deb6aacf591f5b3d1d"
server_key_iv: "e58da952bb09efd0a32c0483bf27ff81"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.290614 18405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:07.325379 18405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:07.327239 18405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:07.327394 18405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:07.331526 18405 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:39453
I20260812 03:48:07.331566 18511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:39453 every 8 connection(s)
I20260812 03:48:07.332017 18405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
I20260812 03:48:07.333024 18512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:07.334554 18512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:48:07.336179 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 18405
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.189@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 03:48:07.344583 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.335321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40968 (local address 127.7.132.190:40737)
0812 03:48:07.335409 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.335412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.335631 (+ 219us) server_negotiation.cc:407] Connection header received
0812 03:48:07.336138 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.336139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.336149 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.336194 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.337088 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.337622 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.338635 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.338907 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.338992 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.341543 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.341545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.341545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.341554 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.343029 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.343535 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.343536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.343537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.343602 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.343883 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.343884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.343884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.344044 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.344126 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.344401 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.344478 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:48:07.344762 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.335308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:40968)
0812 03:48:07.335515 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.335549 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.336033 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.336290 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.336294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.336685 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.336932 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.336939 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.337754 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.337755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.338538 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.338540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.339002 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.339003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.339124 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.339674 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.339682 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.341448 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.343123 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.343130 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.343132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.343454 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.343688 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.343689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.343689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.343803 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.344118 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.344125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.344264 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.344377 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.344523 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":122,"threads_started":1}
I20260812 03:48:07.345582 18512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:48:07.346127 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.336891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58782 (local address 127.7.132.189:39453)
0812 03:48:07.337062 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.337066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.337095 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:48:07.337212 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.337218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.337247 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.337353 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.337938 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.338693 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.339412 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.339722 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.339801 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.342682 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.342689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.342689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.342695 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.344302 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.345087 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.345089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.345090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.345164 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.345428 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.345429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.345429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.345584 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.345718 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.345849 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.345928 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.189@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 03:48:07.352213 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.345833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.0.0.1:58784)
0812 03:48:07.345991 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.346001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.346047 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.346178 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.346178 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.346333 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.346384 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.346390 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.347142 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.347142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.347675 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.347677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.348004 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.348004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.348051 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.348348 (+ 297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.348351 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.349426 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.351422 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.351423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.351423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.351581 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.351751 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.351752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.351752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.351802 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.351995 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.351996 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.352037 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.352086 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.352157 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260812 03:48:07.352383 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.345877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58784 (local address 127.7.132.189:39453)
0812 03:48:07.346018 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.346020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.346031 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:48:07.346088 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.346088 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.346097 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.346167 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.346482 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.347078 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.347750 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.347943 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.347993 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.349495 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.349496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.349496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.349502 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.351351 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.351644 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.351645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.351645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.351692 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.351856 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.351857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.351857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.351943 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.351991 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.352101 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.352201 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260812 03:48:07.353446 18512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Bootstrap starting.
I20260812 03:48:07.354032 18512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:07.354312 18512 log.cc:824] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:07.355012 18512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: No bootstrap required, opened a new log
I20260812 03:48:07.356326 18512 raft_consensus.cc:348] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } }
I20260812 03:48:07.356432 18512 raft_consensus.cc:374] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:07.356474 18512 raft_consensus.cc:729] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d528fb1a1c3f46229b4533c6386ed33b, State: Initialized, Role: FOLLOWER
I20260812 03:48:07.356588 18512 consensus_queue.cc:260] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } }
I20260812 03:48:07.356849 18512 sys_catalog.cc:564] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:07.356820 18520 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } } }
I20260812 03:48:07.356909 18520 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:48:07.358371 18539 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:48:07.358461 18539 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:48:07.360173 18405 master_runner.cc:428] Detected that this master d528fb1a1c3f46229b4533c6386ed33b is joining an existing cluster
I20260812 03:48:07.360276 18405 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.132.189:39453
I20260812 03:48:07.367633 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.361198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.0.0.1:58786)
0812 03:48:07.361465 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.361473 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.361533 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.361686 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.361687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.361931 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.361988 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.361990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.362876 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.362876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.363525 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.363527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.363984 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.363985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.364053 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.364445 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.364450 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.364890 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.366514 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.366515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.366516 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.366672 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.366943 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.366944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.366945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.367000 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.367248 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.367248 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.367286 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.367387 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.367490 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":49,"thread_start_us":109,"threads_started":1}
I20260812 03:48:07.367635 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.361392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58786 (local address 127.7.132.189:39453)
0812 03:48:07.361422 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.361423 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.361522 (+ 99us) server_negotiation.cc:407] Connection header received
0812 03:48:07.361601 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.361602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.361610 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.361635 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.362066 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.362797 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.363617 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.363882 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.363954 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.364981 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.364982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.364982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.364989 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.366417 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.366778 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.366779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.366780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.366861 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.367071 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.367072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.367073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.367182 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.367255 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.367409 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.367540 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":15}
I20260812 03:48:07.368059 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.361204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40984 (local address 127.7.132.190:40737)
0812 03:48:07.361259 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.361261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.361429 (+ 168us) server_negotiation.cc:407] Connection header received
0812 03:48:07.361500 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.361501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.361511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.361569 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.361956 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.362741 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.363786 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.364049 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.364120 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.365125 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.365127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.365127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.365134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.366665 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.367165 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.367166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.367167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.367224 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.367433 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.367433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.367434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.367554 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.367596 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.367859 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.367934 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:48:07.368482 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.361185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:40984)
0812 03:48:07.361350 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.361360 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.361441 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.361574 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.361575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.361814 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.361878 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.361879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.362829 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.362830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.363692 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.363694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.364121 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.364122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.364212 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.364603 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.364608 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.365039 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.366794 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.366795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.366795 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.367072 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.367301 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.367302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.367302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.367360 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.367644 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.367645 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.367706 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.368260 (+ 554us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.368357 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":20,"thread_start_us":113,"threads_started":1}
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 03:48:07.377722 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.370831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.7.132.189:39453)
0812 03:48:07.370891 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.370892 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.371039 (+ 147us) server_negotiation.cc:407] Connection header received
0812 03:48:07.371126 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.371126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.371133 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.371184 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.371657 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.372124 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.373227 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.373384 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.373426 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.375369 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.375371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.375371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.375378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.376591 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.376944 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.376945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.376946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.376999 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.377195 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.377196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.377196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.377343 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.377409 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.377575 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.377647 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:48:07.377750 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.370796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.0.0.1:58802)
0812 03:48:07.370942 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.370983 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.371056 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.371191 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.371194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.371519 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.371581 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.371583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.372208 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.372209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.373132 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.373134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.373449 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.373450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.373529 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.373894 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.373899 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.375278 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.376682 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.376689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.376691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.376871 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.377068 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.377069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.377069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.377126 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.377405 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.377413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.377458 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.377550 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.377660 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260812 03:48:07.378294 17703 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.132.189:39453: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b"
member_type: NON_VOTER
last_known_addr {
host: "127.7.132.189"
port: 39453
}
attrs {
promote: true
}
}
dest_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1"
cas_config_opid_index: -1
I20260812 03:48:07.378660 17703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } }
I20260812 03:48:07.379204 18556 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d528fb1a1c3f46229b4533c6386ed33b (127.7.132.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } }
I20260812 03:48:07.379525 18469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:07.380410 18469 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Refusing update from remote peer 2be07f7ce6c84cf286e210ee9e2ac3c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 03:48:07.380707 18557 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:07.381418 18556 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.132.189:39453
I20260812 03:48:07.381703 18405 master.cc:537] Master@127.7.132.189:39453 shutting down...
I20260812 03:48:07.382174 18557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } }
I20260812 03:48:07.382256 18557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:07.383029 18469 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d528fb1a1c3f46229b4533c6386ed33b (127.7.132.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } }
I20260812 03:48:07.383883 18564 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } }
I20260812 03:48:07.383973 18564 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: LEARNER
I20260812 03:48:07.384099 18520 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be07f7ce6c84cf286e210ee9e2ac3c1. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } }
I20260812 03:48:07.384186 18520 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: LEARNER
I20260812 03:48:07.384248 18562 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } }
I20260812 03:48:07.384285 18562 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: LEARNER
I20260812 03:48:07.385248 18405 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 LEARNER]: Raft consensus shutting down.
I20260812 03:48:07.385344 18405 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 LEARNER]: Raft consensus is shut down!
I20260812 03:48:07.385381 18405 tablet_replica.cc:330] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: stopping tablet replica
I20260812 03:48:07.404989 18405 master.cc:552] Master@127.7.132.189:39453 shutdown complete.
I20260812 03:48:07.405055 18405 master_runner.cc:305] Clearing existing system tablet
I20260812 03:48:07.405191 18405 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:07.405254 18405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:07.405881 18405 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260812 03:48:07.406404 18573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.406574 18405 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:07.406672 18405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.406698 18405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
uuid: "d528fb1a1c3f46229b4533c6386ed33b"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "482f2dea45f5d4deb6aacf591f5b3d1d"
server_key_iv: "e58da952bb09efd0a32c0483bf27ff81"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.408215 18405 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:07.409708 18405 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:48:07.409787 18405 log.cc:1196] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 03:48:07.410018 18405 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Deleting consensus metadata
I20260812 03:48:07.410142 18405 master_runner.cc:315] Copying system tablet from 127.7.132.190:40737
I20260812 03:48:07.410823 18405 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: tablet copy: Beginning tablet copy session from remote peer at address 127.7.132.190:40737
I20260812 03:48:07.417011 18582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.411162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:40986)
0812 03:48:07.411285 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.411295 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.411364 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.411518 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.411518 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.411668 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.411723 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.411724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.412679 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.412680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.413184 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.413186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.413644 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.413645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.413694 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.414164 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.414169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.414473 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.416101 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.416102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.416103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.416282 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.416480 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.416481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.416481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.416531 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.416769 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.416769 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.416802 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.416858 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.416943 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":84,"threads_started":1}
I20260812 03:48:07.417068 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.411258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.7.132.190:40737)
0812 03:48:07.411311 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.411313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.411351 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:48:07.411424 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.411425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.411436 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.411515 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.411792 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.412598 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.413262 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.413562 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.413638 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.414572 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.414574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.414574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.414582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.416020 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.416348 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.416349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.416350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.416403 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.416592 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.416593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.416593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.416709 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.416775 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.416898 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.416985 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:48:07.417572 17731 tablet_copy_service.cc:140] P 2be07f7ce6c84cf286e210ee9e2ac3c1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d528fb1a1c3f46229b4533c6386ed33b ({username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:40986)
I20260812 03:48:07.417708 17731 tablet_copy_service.cc:161] P 2be07f7ce6c84cf286e210ee9e2ac3c1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d528fb1a1c3f46229b4533c6386ed33b at {username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:40986: session id = d528fb1a1c3f46229b4533c6386ed33b-00000000000000000000000000000000
I20260812 03:48:07.418370 17731 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:48:07.419014 18405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:48:07.420101 18405 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: tablet copy: Starting download of 0 data blocks...
I20260812 03:48:07.420257 18405 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: tablet copy: Starting download of 1 WAL segments...
I20260812 03:48:07.421164 18405 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:48:07.422019 17731 tablet_copy_service.cc:342] P 2be07f7ce6c84cf286e210ee9e2ac3c1: Request end of tablet copy session d528fb1a1c3f46229b4533c6386ed33b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.132.189@KRBTEST.COM'} at 127.0.0.1:40986
I20260812 03:48:07.422082 17731 tablet_copy_service.cc:433] P 2be07f7ce6c84cf286e210ee9e2ac3c1: ending tablet copy session d528fb1a1c3f46229b4533c6386ed33b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d528fb1a1c3f46229b4533c6386ed33b
I20260812 03:48:07.424582 18405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:07.425737 18585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.425745 18584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.427629 18587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:07.427711 18405 server_base.cc:1034] running on GCE node
I20260812 03:48:07.427858 18405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:07.427995 18405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:07.429147 18405 hybrid_clock.cc:648] HybridClock initialized: now 1786506487429135 us; error 22 us; skew 500 ppm
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:07.430434 18405 init.cc:377] Logged in from keytab as oryx/127.7.132.189@KRBTEST.COM (short username oryx)
I20260812 03:48:07.430770 18405 webserver.cc:460] Webserver started at http://127.7.132.189:37437/ using document root <none> and password file <none>
I20260812 03:48:07.430886 18405 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:07.430910 18405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:07.431643 18405 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.432199 18596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.432341 18405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:07.432390 18405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
uuid: "d528fb1a1c3f46229b4533c6386ed33b"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "482f2dea45f5d4deb6aacf591f5b3d1d"
server_key_iv: "e58da952bb09efd0a32c0483bf27ff81"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.432452 18405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:07.454833 18405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:07.456308 18405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:07.459388 18405 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:39453
I20260812 03:48:07.459408 18669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:39453 every 8 connection(s)
I20260812 03:48:07.459757 18405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
I20260812 03:48:07.460556 18672 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:48:07.460997 18672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Bootstrap starting.
I20260812 03:48:07.462913 18672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:48:07.463137 18672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: Bootstrap complete.
I20260812 03:48:07.463393 18672 raft_consensus.cc:348] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } }
I20260812 03:48:07.463460 18672 raft_consensus.cc:729] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d528fb1a1c3f46229b4533c6386ed33b, State: Initialized, Role: LEARNER
I20260812 03:48:07.463486 18672 consensus_queue.cc:260] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } }
I20260812 03:48:07.463681 18672 sys_catalog.cc:564] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:07.463702 18675 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } }
I20260812 03:48:07.463820 18675 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: LEARNER
I20260812 03:48:07.465143 18695 catalog_manager.cc:1259] Loaded cluster ID: 00a4a174bbbb4bd0a09cf3a8eab9f15c
I20260812 03:48:07.465191 18695 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: loading cluster ID for follower catalog manager: success
I20260812 03:48:07.466054 18695 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: acquiring CA information for follower catalog manager: success
I20260812 03:48:07.466337 18695 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:48:07.503933 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.498189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.7.132.189:39453)
0812 03:48:07.498323 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.498325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.498334 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:07.498359 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.498360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.498368 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.498437 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.499012 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.499622 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.500363 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.500537 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.500586 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.501700 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.501702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.501702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.501708 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.503030 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.503329 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.503330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.503330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.503380 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.503534 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.503535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.503535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.503628 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.503698 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.503808 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.503865 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.7.132.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.188.keytab.
Entry for principal oryx/127.7.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.188.keytab.
I20260812 03:48:07.527639 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:44135
--webserver_interface=127.7.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.188.keytab
--principal=oryx/127.7.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40737,127.7.132.189:39453,127.7.132.188:44135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:07.610960 18705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:07.611343 18705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:07.611423 18705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:07.612659 18705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:07.612701 18705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:07.612716 18705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:07.612766 18705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:07.612790 18705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:07.614324 18705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40737,127.7.132.189:39453,127.7.132.188:44135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.188.keytab
--principal=oryx/127.7.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:44135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:07.614576 18705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:07.614863 18705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:48:07.619638 18705 server_base.cc:1034] running on GCE node
W20260812 03:48:07.619611 18719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.619587 18712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.619601 18711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:07.619959 18705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:07.620308 18705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:07.621469 18705 hybrid_clock.cc:648] HybridClock initialized: now 1786506487621460 us; error 40 us; skew 500 ppm
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:07.623968 18705 init.cc:377] Logged in from keytab as oryx/127.7.132.188@KRBTEST.COM (short username oryx)
I20260812 03:48:07.624409 18705 webserver.cc:460] Webserver started at http://127.7.132.188:45291/ using document root <none> and password file <none>
I20260812 03:48:07.624616 18705 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:07.624679 18705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:07.624821 18705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:07.625818 18705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data/instance:
uuid: "0a27b760894d4ba892af74b7a05e7ee6"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "d596ddfa577d34f141e669d0a964615c"
server_key_iv: "83af77a7a0d98c11feda28e98c9be0a4"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.626179 18705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal/instance:
uuid: "0a27b760894d4ba892af74b7a05e7ee6"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "d596ddfa577d34f141e669d0a964615c"
server_key_iv: "83af77a7a0d98c11feda28e98c9be0a4"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.627565 18705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:07.628280 18729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.628448 18705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:07.628554 18705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
uuid: "0a27b760894d4ba892af74b7a05e7ee6"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "d596ddfa577d34f141e669d0a964615c"
server_key_iv: "83af77a7a0d98c11feda28e98c9be0a4"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.628634 18705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:07.655799 18705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:07.657424 18705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:07.657620 18705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:07.662925 18705 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:44135
I20260812 03:48:07.662984 18814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:44135 every 8 connection(s)
I20260812 03:48:07.663341 18705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
I20260812 03:48:07.664341 18815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:07.665853 18815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:07.673326 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 18705
I20260812 03:48:07.674944 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.666512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.7.132.190:40737)
0812 03:48:07.666556 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.666559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.666820 (+ 261us) server_negotiation.cc:407] Connection header received
0812 03:48:07.667320 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.667321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.667334 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.667383 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.668085 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.668610 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.669676 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.669864 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.669942 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.672191 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.672192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.672192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.672199 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.673509 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.673961 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.673963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.673963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.674028 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.674291 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.674292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.674292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.674409 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.674487 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.674777 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.674858 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:48:07.675244 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.666467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:40996)
0812 03:48:07.666672 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.666750 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.667211 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.667451 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.667454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.667750 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.667970 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.667974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.668707 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.668709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.669580 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.669582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.669967 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.669968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.670092 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.670596 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.670604 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.672104 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.673588 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.673595 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.673596 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.673834 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.674103 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.674104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.674104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.674213 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.674478 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.674485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.674621 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.674758 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.674910 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":105,"threads_started":1}
I20260812 03:48:07.675771 18815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 03:48:07.682946 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.676024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.0.0.1:58844)
0812 03:48:07.676073 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.676083 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.676115 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.676250 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.676251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.676393 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.676439 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.676441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.677091 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.677092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.677804 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.677807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.678273 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.678274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.678351 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.678647 (+ 296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.678652 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.680138 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.681853 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.681854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.681854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.682052 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.682273 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.682273 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.682274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.682331 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.682624 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.682625 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.682667 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.682776 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.682864 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:48:07.682955 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.676084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58844 (local address 127.7.132.189:39453)
0812 03:48:07.676123 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.676124 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.676150 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:48:07.676182 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.676182 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.676192 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.676230 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.676510 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.676959 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.677907 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.678189 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.678261 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.680220 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.680221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.680222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.680227 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.681750 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.682145 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.682146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.682146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.682204 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.682406 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.682407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.682408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.682554 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.682632 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.682801 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.682874 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:48:07.683461 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.673923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.7.132.188:44135)
0812 03:48:07.674078 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.674083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.674168 (+ 85us) server_negotiation.cc:407] Connection header received
0812 03:48:07.674239 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.674242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.674259 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.674337 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.674974 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.675845 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.676798 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.677162 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.677255 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.680316 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.680323 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.680323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.680329 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.681969 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.682317 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.682320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.682320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.682398 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.682650 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.682650 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.682651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.682848 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.682978 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.683341 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.683398 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260812 03:48:07.683499 18815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
I20260812 03:48:07.688928 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.683738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.0.0.1:58338)
0812 03:48:07.683776 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.683784 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.683814 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.683929 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.683929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.684051 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.684079 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.684080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.684627 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.684628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.685054 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.685055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.685372 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.685373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.685404 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.685683 (+ 279us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.685686 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.686746 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.688258 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.688259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.688259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.688397 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.688536 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.688537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.688537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.688581 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.688737 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.688737 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.688766 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.688812 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.688876 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 03:48:07.689056 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.683790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.7.132.188:44135)
0812 03:48:07.683825 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.683826 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.683835 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:07.683854 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.683855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.683864 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.683928 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.684150 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.684565 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.685135 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.685314 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.685361 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.686837 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.686838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.686839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.686844 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.688191 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.688448 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.688448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.688449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.688488 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.688616 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.688617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.688617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.688690 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.688732 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.688826 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.688916 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":18}
I20260812 03:48:07.690171 18815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Bootstrap starting.
I20260812 03:48:07.691066 18815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:07.691440 18815 log.cc:824] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:07.692173 18815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: No bootstrap required, opened a new log
I20260812 03:48:07.693602 18815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } }
I20260812 03:48:07.693712 18815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:07.693733 18815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a27b760894d4ba892af74b7a05e7ee6, State: Initialized, Role: FOLLOWER
I20260812 03:48:07.693872 18815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } }
I20260812 03:48:07.694118 18821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } } }
I20260812 03:48:07.694160 18815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:07.694226 18821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:48:07.695833 18838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:48:07.695883 18838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:48:07.697989 18705 master_runner.cc:428] Detected that this master 0a27b760894d4ba892af74b7a05e7ee6 is joining an existing cluster
I20260812 03:48:07.698047 18705 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.132.188:44135
I20260812 03:48:07.722667 18848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.699370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41006)
0812 03:48:07.699538 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.699548 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.699643 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.699844 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.699846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.700040 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.700118 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.700120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.700838 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.700838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.701580 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.701582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.702095 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.702095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.702168 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.702620 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.702626 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.719487 (+ 16861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.721447 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.721448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.721449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.721730 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.721939 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.721940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.721940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.721999 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.722300 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.722300 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.722363 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.722436 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.722542 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":116,"threads_started":1}
I20260812 03:48:07.722983 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.699377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41006 (local address 127.7.132.190:40737)
0812 03:48:07.699424 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.699425 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.699615 (+ 190us) server_negotiation.cc:407] Connection header received
0812 03:48:07.699729 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.699730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.699741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.699802 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.700216 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.700766 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.701672 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.701964 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.702003 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.719630 (+ 17627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.719632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.719632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.719643 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.721338 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.721813 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.721814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.721814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.721871 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.722073 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.722074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.722075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.722214 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.722274 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.722810 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.722890 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:48:07.723492 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.699647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.7.132.188:44135)
0812 03:48:07.699682 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.699684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.702873 (+ 3189us) server_negotiation.cc:407] Connection header received
0812 03:48:07.702904 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.702904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.702913 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.702970 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.703319 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.703706 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.704625 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.704803 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.704848 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.719684 (+ 14836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.719685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.719685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.719692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.722228 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.722575 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.722576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.722576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.722638 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.722864 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.722865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.722865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.722978 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.723013 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.723335 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.723406 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:48:07.723492 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.699891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.7.132.189:39453)
0812 03:48:07.699930 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.699931 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.699940 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:07.699961 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.699962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.699970 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.700032 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.700401 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.701114 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.702098 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.702424 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.702516 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.719696 (+ 17180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.719698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.719698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.719707 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.721633 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.722025 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.722026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.722027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.722082 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.722309 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.722310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.722310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.722458 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.722501 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.722710 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.723407 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:48:07.723803 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.699624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.0.0.1:58346)
0812 03:48:07.702795 (+ 3171us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.702805 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.702864 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.702987 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.702988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.703175 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.703236 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.703238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.703784 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.703786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.704544 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.704547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.704873 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.704874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.704940 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.705726 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.705731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.719575 (+ 13844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.722304 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.722305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.722305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.722506 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.722724 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.722725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.722725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.722790 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.723048 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.723049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.723076 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.723556 (+ 480us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.723657 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3155,"mutex_wait_us":1,"thread_start_us":147,"threads_started":1}
I20260812 03:48:07.724573 18853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.699384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.0.0.1:58848)
0812 03:48:07.699824 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.699833 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.699871 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.700045 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.700046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.700248 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.700320 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.700321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.701200 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.701201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.702010 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.702012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.702528 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.702529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.702596 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.703030 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.703033 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.719569 (+ 16536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.721728 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.721729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.721729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.721934 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.722157 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.722158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.722158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.722221 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.722551 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.722552 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.722592 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.723939 (+ 1347us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.724507 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":101,"threads_started":1}
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506486, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
I20260812 03:48:07.731281 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.724906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.0.0.1:58354)
0812 03:48:07.724946 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.724955 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.724996 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.725162 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.725162 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.725308 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.725350 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.725351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.726080 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.726081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.726903 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.726906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.727209 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.727211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.727288 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.727602 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.727607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.728850 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.730227 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.730227 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.730228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.730384 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.730564 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.730565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.730565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.730618 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.730955 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.730956 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.731017 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.731085 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.731175 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 03:48:07.731282 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.724958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.7.132.188:44135)
0812 03:48:07.724995 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.724997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.725006 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:07.725066 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.725066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.725075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.725118 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.725432 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.726007 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.726975 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.727137 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.727188 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.728930 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.728931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.728932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.728938 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.730127 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.730450 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.730451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.730452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.730503 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.730739 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.730740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.730740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.730887 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.730950 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.731107 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.731175 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:48:07.731832 17703 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.132.188:44135: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6"
member_type: NON_VOTER
last_known_addr {
host: "127.7.132.188"
port: 44135
}
attrs {
promote: true
}
}
dest_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1"
cas_config_opid_index: 5
I20260812 03:48:07.732086 17703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } }
I20260812 03:48:07.732493 18556 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.732928 18769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:07.733295 18556 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.132.188:44135
I20260812 03:48:07.733369 18556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.733439 18556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:07.733850 18769 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Refusing update from remote peer 2be07f7ce6c84cf286e210ee9e2ac3c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:48:07.733999 18705 master.cc:537] Master@127.7.132.188:44135 shutting down...
I20260812 03:48:07.734499 18557 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:48:07.735260 17673 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260812 03:48:07.737167 18705 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:07.737262 18705 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:07.737325 18705 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: stopping tablet replica
I20260812 03:48:07.738655 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.732566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.0.0.1:58858)
0812 03:48:07.732618 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.732629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.732674 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.735429 (+ 2755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.735430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.735432 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:07.735463 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.735464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.736312 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.736313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.737511 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.737515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.738305 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.738307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.738496 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.738516 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.738563 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.738594 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:48:07.738837 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.732695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58858 (local address 127.7.132.189:39453)
0812 03:48:07.734403 (+ 1708us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.734405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.734414 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:07.734436 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.734437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.734462 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.734523 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:07.735543 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.736226 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.737600 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.738147 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.738211 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.738274 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.738745 (+ 471us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.738780 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1668}
I20260812 03:48:07.739213 18634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 LEARNER]: Refusing update from remote peer 2be07f7ce6c84cf286e210ee9e2ac3c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:48:07.739318 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.739238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.0.0.1:58368)
0812 03:48:07.739269 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.739297 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 03:48:07.739401 17673 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 -> Peer 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188:44135): Couldn't send request to peer 0a27b760894d4ba892af74b7a05e7ee6. Status: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:48:07.739560 18557 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 03:48:07.739933 18634 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.740149 18705 master.cc:552] Master@127.7.132.188:44135 shutdown complete.
I20260812 03:48:07.740180 18705 master_runner.cc:305] Clearing existing system tablet
I20260812 03:48:07.740293 18705 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:07.740327 18705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:07.740952 18858 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.741040 18858 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: LEARNER
I20260812 03:48:07.740984 18675 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be07f7ce6c84cf286e210ee9e2ac3c1. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: NON_VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: true } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.741078 18675 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: LEARNER
I20260812 03:48:07.741405 18705 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:48:07.741662 18557 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: attempting to promote NON_VOTER d528fb1a1c3f46229b4533c6386ed33b to VOTER
I20260812 03:48:07.741874 18557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } }
I20260812 03:48:07.741991 18859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.742106 18705 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:07.742146 18705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.742168 18705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
uuid: "0a27b760894d4ba892af74b7a05e7ee6"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "d596ddfa577d34f141e669d0a964615c"
server_key_iv: "83af77a7a0d98c11feda28e98c9be0a4"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.742385 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.742253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.0.0.1:58384)
0812 03:48:07.742328 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.742350 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 03:48:07.742527 18634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 LEARNER]: Refusing update from remote peer 2be07f7ce6c84cf286e210ee9e2ac3c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 03:48:07.742942 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.742822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.0.0.1:58398)
0812 03:48:07.742873 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.742899 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 03:48:07.742892 18860 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 03:48:07.742934 18705 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:48:07.743088 17673 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 -> Peer 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188:44135): Couldn't send request to peer 0a27b760894d4ba892af74b7a05e7ee6. Status: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44135: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:48:07.743559 18556 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d528fb1a1c3f46229b4533c6386ed33b (127.7.132.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.744197 18705 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:48:07.744263 18705 log.cc:1196] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 03:48:07.744251 18556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.744323 18556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:07.744377 18556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.744421 18556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:07.744442 18705 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Deleting consensus metadata
I20260812 03:48:07.744606 18705 master_runner.cc:315] Copying system tablet from 127.7.132.190:40737
I20260812 03:48:07.744619 18634 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d528fb1a1c3f46229b4533c6386ed33b (127.7.132.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.745162 18705 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: tablet copy: Beginning tablet copy session from remote peer at address 127.7.132.190:40737
I20260812 03:48:07.745308 18675 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.745393 18675 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:48:07.750998 18869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.745566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41014)
0812 03:48:07.745692 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.745700 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.745751 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.745866 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.745866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.745997 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.746038 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.746039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.746850 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.746851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.747389 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.747391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.747807 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.747807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.747853 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.748249 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.748252 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.748564 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.750017 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.750018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.750019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.750160 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.750360 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.750361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.750361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.750442 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.750680 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.750681 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.750812 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.750864 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.750930 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260812 03:48:07.751065 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.745605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.7.132.190:40737)
0812 03:48:07.745639 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.745641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.745760 (+ 119us) server_negotiation.cc:407] Connection header received
0812 03:48:07.745798 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.745799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.745808 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.745863 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.746101 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.746787 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.747459 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.747726 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.747800 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.748632 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.748633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.748633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.748639 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.749934 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.750223 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.750224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.750224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.750280 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.750504 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.750505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.750506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.750619 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.750683 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.750914 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.750983 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:48:07.751493 17731 tablet_copy_service.cc:140] P 2be07f7ce6c84cf286e210ee9e2ac3c1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0a27b760894d4ba892af74b7a05e7ee6 ({username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:41014)
I20260812 03:48:07.751550 17731 tablet_copy_service.cc:161] P 2be07f7ce6c84cf286e210ee9e2ac3c1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0a27b760894d4ba892af74b7a05e7ee6 at {username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:41014: session id = 0a27b760894d4ba892af74b7a05e7ee6-00000000000000000000000000000000
I20260812 03:48:07.752112 17731 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:48:07.752581 18705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:48:07.753706 18705 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: tablet copy: Starting download of 0 data blocks...
I20260812 03:48:07.753841 18705 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: tablet copy: Starting download of 1 WAL segments...
I20260812 03:48:07.754604 18705 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:48:07.755347 17731 tablet_copy_service.cc:342] P 2be07f7ce6c84cf286e210ee9e2ac3c1: Request end of tablet copy session 0a27b760894d4ba892af74b7a05e7ee6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:41014
I20260812 03:48:07.755401 17731 tablet_copy_service.cc:433] P 2be07f7ce6c84cf286e210ee9e2ac3c1: ending tablet copy session 0a27b760894d4ba892af74b7a05e7ee6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0a27b760894d4ba892af74b7a05e7ee6
I20260812 03:48:07.757643 18705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:07.758828 18872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.758873 18874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:07.758961 18705 server_base.cc:1034] running on GCE node
W20260812 03:48:07.759126 18871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:07.759249 18705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:07.759371 18705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:07.760481 18705 hybrid_clock.cc:648] HybridClock initialized: now 1786506487760481 us; error 32 us; skew 500 ppm
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:07.761636 18705 init.cc:377] Logged in from keytab as oryx/127.7.132.188@KRBTEST.COM (short username oryx)
I20260812 03:48:07.761884 18705 webserver.cc:460] Webserver started at http://127.7.132.188:38029/ using document root <none> and password file <none>
I20260812 03:48:07.762015 18705 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:07.762063 18705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:07.762660 18705 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.763161 18889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.763293 18705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.763360 18705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
uuid: "0a27b760894d4ba892af74b7a05e7ee6"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "d596ddfa577d34f141e669d0a964615c"
server_key_iv: "83af77a7a0d98c11feda28e98c9be0a4"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.763433 18705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:07.769810 18705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:07.771476 18705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:07.774108 18705 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:44135
I20260812 03:48:07.774533 18705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
I20260812 03:48:07.774590 18963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:44135 every 8 connection(s)
I20260812 03:48:07.775362 18964 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:48:07.775839 18964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Bootstrap starting.
I20260812 03:48:07.778671 18964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:48:07.778931 18964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Bootstrap complete.
I20260812 03:48:07.779158 18964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } }
I20260812 03:48:07.779246 18964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0a27b760894d4ba892af74b7a05e7ee6, State: Initialized, Role: LEARNER
I20260812 03:48:07.779284 18964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } }
I20260812 03:48:07.779446 18964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:07.779455 18972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:07.779613 18972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:48:07.780964 18989 catalog_manager.cc:1259] Loaded cluster ID: 00a4a174bbbb4bd0a09cf3a8eab9f15c
I20260812 03:48:07.781051 18989 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: loading cluster ID for follower catalog manager: success
I20260812 03:48:07.782022 18989 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: acquiring CA information for follower catalog manager: success
I20260812 03:48:07.782287 18989 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:48:07.797832 18990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.791762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58412 (local address 127.7.132.188:44135)
0812 03:48:07.791895 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.791897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.791903 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:48:07.791922 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.791923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.791931 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.792006 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.792536 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.793158 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.793881 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.794067 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.794137 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.795341 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.795342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.795343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.795350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.796736 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.797078 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.797080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.797080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.797141 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.797339 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.797340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.797340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.797442 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.797544 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.797644 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.797720 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260812 03:48:07.798353 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 17789
I20260812 03:48:07.818869 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:39265
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:40737
--webserver_port=37857
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453,127.7.132.188:44135 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:07.899996 18991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:07.900190 18991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:07.900211 18991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:07.901330 18991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:07.901383 18991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:07.901409 18991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:07.902978 18991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:39265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=37857
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453,127.7.132.188:44135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:07.903187 18991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:07.903436 18991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:07.908402 19006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.908412 19007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:07.908412 19010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:07.908712 18991 server_base.cc:1034] running on GCE node
I20260812 03:48:07.908941 18991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:07.909186 18991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:07.910326 18991 hybrid_clock.cc:648] HybridClock initialized: now 1786506487910319 us; error 32 us; skew 500 ppm
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:07.912832 18991 init.cc:377] Logged in from keytab as oryx/127.7.132.129@KRBTEST.COM (short username oryx)
I20260812 03:48:07.913260 18991 webserver.cc:460] Webserver started at http://127.7.132.129:37857/ using document root <none> and password file <none>
I20260812 03:48:07.913527 18991 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:07.913602 18991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:07.914929 18991 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:48:07.915486 19021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.915654 18991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.915757 18991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "ee97fa4ed1794eb09fdba3945f17ac97"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "b78531ef7e8b00e889219fd2bc8b7fd4"
server_key_iv: "a75442456b464168e5961841688e7173"
server_key_version: "encryptionkey@0"
I20260812 03:48:07.916069 18991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:07.938961 18991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:07.940780 18991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:07.940943 18991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:07.941174 18991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:07.941512 18991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:07.941566 18991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.941609 18991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:07.941636 18991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:07.948061 18991 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:39265
I20260812 03:48:07.948107 19197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:39265 every 8 connection(s)
I20260812 03:48:07.948422 18991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:48:07.954118 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 18991
I20260812 03:48:07.954303 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 17982
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 03:48:07.960760 18990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.949671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:35041 (local address 127.7.132.188:44135)
0812 03:48:07.949711 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.949712 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.949874 (+ 162us) server_negotiation.cc:407] Connection header received
0812 03:48:07.950725 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.950726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.950743 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.950797 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.951523 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.952296 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.953493 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.953783 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.953834 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.956173 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.956174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.956175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.956182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.959150 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.959654 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.959656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.959656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.959715 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.959991 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.959992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.959992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.960132 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.960175 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.960551 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.960646 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:48:07.961311 19205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.949571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.7.132.129:35041)
0812 03:48:07.949816 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.949824 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.950579 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.950883 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.950888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.951281 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.951407 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.951413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.952393 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.952394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.953399 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.953402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.953856 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.953857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.953940 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.954371 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.954381 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.956055 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.959252 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.959259 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.959261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.959578 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.959787 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.959787 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.959787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.959916 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.960226 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.960244 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.960395 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.960886 (+ 491us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.961050 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":22,"thread_start_us":72,"threads_started":1}
I20260812 03:48:07.962803 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.949109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.7.132.129:58781)
0812 03:48:07.949703 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.949712 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.950571 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.950914 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.950915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.951236 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.951407 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.951413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.952393 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.952394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.953399 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.953402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.954341 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.954343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.954414 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.957940 (+ 3526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.957945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.959556 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.961713 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.961714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.961714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.961875 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.962121 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.962122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.962122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.962182 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.962436 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.962437 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.962487 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.962563 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.962671 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":146,"thread_start_us":254,"threads_started":1}
I20260812 03:48:07.962814 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.949118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:58781 (local address 127.7.132.189:39453)
0812 03:48:07.949192 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.949194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.949771 (+ 577us) server_negotiation.cc:407] Connection header received
0812 03:48:07.950788 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.950790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.950802 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.950850 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.951521 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.952291 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.953907 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.954213 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.954283 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.959656 (+ 5373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.959657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.959657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.959665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.961201 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.961974 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.961976 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.961976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.962037 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.962239 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.962240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.962240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.962359 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.962432 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.962586 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.962709 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:48:07.971701 19198 heartbeater.cc:347] Connected to a master server at 127.7.132.188:44135
I20260812 03:48:07.971817 19198 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:07.972054 19198 heartbeater.cc:511] Master 127.7.132.188:44135 requested a full tablet report, sending...
Aug 12 03:48:07 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506487, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:07.972608 18916 ts_manager.cc:194] Registered new tserver with Master: ee97fa4ed1794eb09fdba3945f17ac97 (127.7.132.129:39265)
I20260812 03:48:07.973330 19201 heartbeater.cc:347] Connected to a master server at 127.7.132.189:39453
I20260812 03:48:07.973393 19201 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:07.973461 19201 heartbeater.cc:511] Master 127.7.132.189:39453 requested a full tablet report, sending...
I20260812 03:48:07.973902 18618 ts_manager.cc:194] Registered new tserver with Master: ee97fa4ed1794eb09fdba3945f17ac97 (127.7.132.129:39265)
I20260812 03:48:07.975926 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.949109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:52777)
0812 03:48:07.949458 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:07.949497 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:07.950571 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:07.950883 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:07.950888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:07.951196 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:07.951406 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.951411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.952133 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.952134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.952950 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:07.952952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.953356 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.953357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:07.953524 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.971336 (+ 17812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:07.971341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:07.972818 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:07.974962 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.974963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.974964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.975128 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.975333 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:07.975334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:07.975334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:07.975381 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:07.975684 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:07.975685 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:07.975719 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 03:48:07.975775 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:48:07.975860 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":1,"thread_start_us":246,"threads_started":1}
I20260812 03:48:07.976058 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:07.949303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:52777 (local address 127.7.132.190:40737)
0812 03:48:07.949373 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:07.949375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:07.949650 (+ 275us) server_negotiation.cc:407] Connection header received
0812 03:48:07.950765 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:07.950767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:07.950778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:07.950828 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:07.951506 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.952009 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.953044 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:07.953270 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:07.953346 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:07.972924 (+ 19578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:07.972926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:07.972927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:07.972936 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:07.974869 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.975199 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.975200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.975201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.975268 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:07.975445 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:07.975446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:07.975446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:07.975603 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:07.975700 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:07.975832 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 03:48:07.975932 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:48:07.976445 19200 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40737
I20260812 03:48:07.976501 19200 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:07.976564 19200 heartbeater.cc:511] Master 127.7.132.190:40737 requested a full tablet report, sending...
I20260812 03:48:07.976864 17703 ts_manager.cc:194] Re-registered known tserver with Master: ee97fa4ed1794eb09fdba3945f17ac97 (127.7.132.129:39265)
I20260812 03:48:07.977224 17703 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:52777
I20260812 03:48:07.978336 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:45091
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:40737
--webserver_port=39853
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453,127.7.132.188:44135 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:08.061031 19206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:08.061326 19206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:08.061365 19206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:08.062541 19206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:08.062604 19206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:08.062647 19206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:08.064438 19206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:45091
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=39853
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453,127.7.132.188:44135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:08.064718 19206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:08.064960 19206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:08.070109 19214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:08.070118 19211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:08.070118 19212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:08.070431 19206 server_base.cc:1034] running on GCE node
I20260812 03:48:08.070605 19206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:08.070856 19206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:08.071996 19206 hybrid_clock.cc:648] HybridClock initialized: now 1786506488071986 us; error 31 us; skew 500 ppm
Aug 12 03:48:08 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506488, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:08.074324 19206 init.cc:377] Logged in from keytab as oryx/127.7.132.130@KRBTEST.COM (short username oryx)
I20260812 03:48:08.074772 19206 webserver.cc:460] Webserver started at http://127.7.132.130:39853/ using document root <none> and password file <none>
I20260812 03:48:08.074965 19206 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:08.075016 19206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:08.076242 19206 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:08.076740 19222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:08.076892 19206 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:08.076982 19206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "18837fec8c8743aab7fe31912772198a"
format_stamp: "Formatted at 2026-08-12 03:48:06 on dist-test-slave-39l8"
server_key: "36d330fd0d0afe3ccfbc3b1af04bfb8e"
server_key_iv: "089c4edcb9d6e7652530772e04ad8a7c"
server_key_version: "encryptionkey@0"
I20260812 03:48:08.077303 19206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:08.088179 19206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:08.090322 19206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:08.090431 19206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:08.090606 19206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:08.090930 19206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:08.090978 19206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:08.090998 19206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:08.091012 19206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:08.097101 19206 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:45091
I20260812 03:48:08.097189 19409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:45091 every 8 connection(s)
I20260812 03:48:08.097530 19206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:08.103945 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 19206
I20260812 03:48:08.104152 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 18202
Aug 12 03:48:08 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506488, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:08.108402 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.098268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:36305 (local address 127.7.132.190:40737)
0812 03:48:08.098321 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.098323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.098595 (+ 272us) server_negotiation.cc:407] Connection header received
0812 03:48:08.099277 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.099278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.099291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.099325 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.100258 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.100987 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.102213 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.102390 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.102476 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.105244 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.105245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.105246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.105253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.106899 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.107377 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.107378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.107378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.107445 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.107706 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.107707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.107707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.107855 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.107930 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.108180 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.108265 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:48:08.108429 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.098277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.130:36305)
0812 03:48:08.098467 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.098507 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.099147 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.099429 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.099432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.099944 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:08.100146 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.100152 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.101065 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.101066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.102128 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.102130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.102485 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.102487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.102635 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.103198 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:08.103209 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:08.105131 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:08.107049 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.107056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.107058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.107299 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.107525 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.107526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.107526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.107638 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.107923 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:08.107928 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:08.108035 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.108142 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.108274 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":15,"spinlock_wait_cycles":7552,"thread_start_us":120,"threads_started":1}
I20260812 03:48:08.108939 19411 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40737
I20260812 03:48:08.109028 19411 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:08.109216 19411 heartbeater.cc:511] Master 127.7.132.190:40737 requested a full tablet report, sending...
I20260812 03:48:08.109618 17703 ts_manager.cc:194] Re-registered known tserver with Master: 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091)
I20260812 03:48:08.109987 17703 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:36305
Aug 12 03:48:08 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506488, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
Aug 12 03:48:08 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506488, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 03:48:08.112167 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:44111
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:40737
--webserver_port=38731
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:40885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453,127.7.132.188:44135 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:48:08.114683 18990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.098581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:42871 (local address 127.7.132.188:44135)
0812 03:48:08.098631 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.098633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.098643 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:08.099276 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.099278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.099291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.099325 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.100258 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.100981 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.102849 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.103046 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.103145 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.111777 (+ 8632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.111779 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.111780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.111789 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.113488 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.113868 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.113870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.113870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.113936 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.114145 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.114146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.114146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.114268 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.114338 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.114501 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.114587 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:48:08.114730 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.098277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.7.132.130:42871)
0812 03:48:08.098461 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.098507 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.099146 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.099428 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.099432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.099937 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:08.100146 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.100158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.102232 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.102233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.102772 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.102773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.103146 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.103147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.103242 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.103644 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:08.103648 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:08.111681 (+ 8033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:08.113586 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.113587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.113587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.113780 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.114007 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.114008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.114008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.114076 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.114341 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:08.114343 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:08.114385 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.114462 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.114594 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":6,"thread_start_us":91,"threads_started":1}
I20260812 03:48:08.115186 19410 heartbeater.cc:347] Connected to a master server at 127.7.132.188:44135
I20260812 03:48:08.115242 19410 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:08.115325 19410 heartbeater.cc:511] Master 127.7.132.188:44135 requested a full tablet report, sending...
I20260812 03:48:08.115640 18916 ts_manager.cc:194] Registered new tserver with Master: 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091)
I20260812 03:48:08.115644 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.098700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:43725 (local address 127.7.132.189:39453)
0812 03:48:08.098792 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.098794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.098803 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:08.099262 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.099263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.099275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.099325 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.100246 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.100721 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.102548 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.103181 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.103258 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.112435 (+ 9177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.112436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.112437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.112445 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.114249 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.114719 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.114721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.114722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.114784 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.115079 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.115080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.115080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.115221 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.115277 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.115479 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.115564 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:48:08.115690 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.098561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.7.132.130:43725)
0812 03:48:08.098678 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.098688 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.099159 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.099670 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.099671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.099925 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:08.100145 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.100149 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.100833 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.100834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.102128 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.102130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.103264 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.103265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.103324 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.103819 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:08.103824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:08.112330 (+ 8506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:08.114325 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.114326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.114327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.114608 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.114946 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.114947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.114947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.115008 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.115328 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:08.115329 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:08.115368 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.115440 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.115569 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":24,"thread_start_us":61,"threads_started":1}
I20260812 03:48:08.116290 19412 heartbeater.cc:347] Connected to a master server at 127.7.132.189:39453
I20260812 03:48:08.116354 19412 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:08.116411 19412 heartbeater.cc:511] Master 127.7.132.189:39453 requested a full tablet report, sending...
I20260812 03:48:08.116675 18618 ts_manager.cc:194] Registered new tserver with Master: 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091)
W20260812 03:48:08.196003 19420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:08.196200 19420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:08.196223 19420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:08.197359 19420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:08.197400 19420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:08.197453 19420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:08.198961 19420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:40885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:44111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=38731
--tserver_master_addrs=127.7.132.190:40737,127.7.132.189:39453,127.7.132.188:44135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:08.199239 19420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:08.199481 19420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:08.204202 19428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:08.204280 19432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:08.204201 19430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:08.204280 19420 server_base.cc:1034] running on GCE node
I20260812 03:48:08.204568 19420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:08.204777 19420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:08.205921 19420 hybrid_clock.cc:648] HybridClock initialized: now 1786506488205908 us; error 29 us; skew 500 ppm
Aug 12 03:48:08 dist-test-slave-39l8 krb5kdc[17629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506488, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:08.208514 19420 init.cc:377] Logged in from keytab as oryx/127.7.132.131@KRBTEST.COM (short username oryx)
I20260812 03:48:08.209059 19420 webserver.cc:460] Webserver started at http://127.7.132.131:38731/ using document root <none> and password file <none>
I20260812 03:48:08.209340 19420 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:08.209403 19420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:08.210631 19420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:08.211445 19439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:08.211659 19420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:08.211781 19420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "038eef5f4c8545f59b60655118824e7e"
format_stamp: "Formatted at 2026-08-12 03:48:07 on dist-test-slave-39l8"
server_key: "a9045df5285fb598f751be607d18608d"
server_key_iv: "0842762f595ba5317e6c86f0ecd8f1e0"
server_key_version: "encryptionkey@0"
I20260812 03:48:08.212246 19420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:08.230657 19420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:08.232609 19420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:08.232717 19420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:08.232920 19420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:08.233291 19420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:08.233343 19420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:08.233382 19420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:08.233405 19420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:08.239686 19420 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:44111
I20260812 03:48:08.239750 19622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:44111 every 8 connection(s)
I20260812 03:48:08.240051 19420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:48:08 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506488, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
Aug 12 03:48:08 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506488, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.188@KRBTEST.COM
I20260812 03:48:08.248764 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 19420
Aug 12 03:48:08 dist-test-slave-39l8 krb5kdc[17629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506488, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.189@KRBTEST.COM
I20260812 03:48:08.252744 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.240790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:55923 (local address 127.7.132.190:40737)
0812 03:48:08.240861 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.240863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.241324 (+ 461us) server_negotiation.cc:407] Connection header received
0812 03:48:08.242519 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.242521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.242533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.242601 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.243425 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.243912 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.245371 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.245673 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.245758 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.248275 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.248277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.248277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.248284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.250762 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.251597 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.251599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.251599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.251662 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.251958 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.251960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.251960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.252113 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.252160 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.252565 (+ 405us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.252646 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:48:08.253022 18990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.241142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48249 (local address 127.7.132.188:44135)
0812 03:48:08.241187 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.241189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.241493 (+ 304us) server_negotiation.cc:407] Connection header received
0812 03:48:08.242454 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.242455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.242465 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.242498 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.243445 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.244172 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.245371 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.245673 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.245748 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.250042 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.250044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.250044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.250052 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.251783 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.252191 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.252193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.252193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.252253 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.252455 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.252456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.252456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.252594 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.252638 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.252855 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.252930 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:48:08.253257 19630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.240742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:55923)
0812 03:48:08.241199 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.241245 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.242308 (+ 1063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.242630 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.242634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.243116 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:08.243315 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.243319 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.244017 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.244018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.245257 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.245260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.246110 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.246111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.246173 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.246609 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:08.246614 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:08.248176 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:08.250860 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.251101 (+ 241us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.251106 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.251497 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.251740 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.251740 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.251741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.251875 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.252210 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:08.252217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:08.252387 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.252843 (+ 456us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.253008 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260812 03:48:08.253352 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.241051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.7.132.131:48249)
0812 03:48:08.241447 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.241455 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.242320 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.242634 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.242635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.243119 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:08.243316 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.243322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.244561 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.244562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.245268 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.245269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.245764 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.245765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.245898 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.246468 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:08.246479 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:08.249924 (+ 3445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:08.251863 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.251864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.251864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.252098 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.252325 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.252325 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.252326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.252381 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.252687 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:08.252688 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:08.252725 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.253129 (+ 404us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.253222 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":82,"threads_started":1}
I20260812 03:48:08.253846 19623 heartbeater.cc:347] Connected to a master server at 127.7.132.188:44135
I20260812 03:48:08.253945 19623 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:08.253842 19624 heartbeater.cc:347] Connected to a master server at 127.7.132.190:40737
I20260812 03:48:08.254127 19623 heartbeater.cc:511] Master 127.7.132.188:44135 requested a full tablet report, sending...
I20260812 03:48:08.254146 19624 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:08.254212 19624 heartbeater.cc:511] Master 127.7.132.190:40737 requested a full tablet report, sending...
I20260812 03:48:08.254329 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.240744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:50907 (local address 127.7.132.189:39453)
0812 03:48:08.240997 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.240999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.241404 (+ 405us) server_negotiation.cc:407] Connection header received
0812 03:48:08.242433 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.242434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.242447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.242503 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.243444 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.244175 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.245981 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.246223 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.246295 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.249364 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.249366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.249366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.249374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.251088 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.253474 (+ 2386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.253476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.253476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.253533 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.253775 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.253776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.253777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.253913 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.253961 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.254166 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.254240 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260812 03:48:08.254490 17703 ts_manager.cc:194] Re-registered known tserver with Master: 038eef5f4c8545f59b60655118824e7e (127.7.132.131:44111)
I20260812 03:48:08.254554 18916 ts_manager.cc:194] Registered new tserver with Master: 038eef5f4c8545f59b60655118824e7e (127.7.132.131:44111)
I20260812 03:48:08.254635 19628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.240742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.7.132.131:50907)
0812 03:48:08.241327 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.241336 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.242308 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.242634 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.242635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.243273 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:08.243321 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.243322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.244253 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.244254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.245899 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.245901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.246309 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.246310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.246367 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.246786 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:08.246791 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:08.249253 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:08.251178 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.251179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.251180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.253411 (+ 2231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.253609 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:08.253610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:08.253610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:08.253668 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:08.254012 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:08.254012 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:08.254051 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.254433 (+ 382us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.254528 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":832,"thread_start_us":90,"threads_started":1}
I20260812 03:48:08.255009 17703 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:55923
I20260812 03:48:08.255131 19625 heartbeater.cc:347] Connected to a master server at 127.7.132.189:39453
I20260812 03:48:08.255184 19625 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:08.255276 19625 heartbeater.cc:511] Master 127.7.132.189:39453 requested a full tablet report, sending...
I20260812 03:48:08.255524 18618 ts_manager.cc:194] Registered new tserver with Master: 038eef5f4c8545f59b60655118824e7e (127.7.132.131:44111)
I20260812 03:48:08.260735 19637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.250404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.7.132.190:40737)
0812 03:48:08.250736 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.250739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.250922 (+ 183us) server_negotiation.cc:407] Connection header received
0812 03:48:08.250943 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.250944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.250955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.251020 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.251892 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.252661 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.255923 (+ 3262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.256126 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.256211 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.257729 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.257731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.257731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.257739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.259436 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.259841 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.259842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.259843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.259904 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.260094 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.260095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.260095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.260218 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.260325 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.260432 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.260516 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":54,"threads_started":1}
I20260812 03:48:08.261801 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.251705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.7.132.189:39453)
0812 03:48:08.252479 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.252481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.252487 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:48:08.252506 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.252507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.252517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.252567 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.253622 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.254414 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.256292 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.256586 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.256650 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.258655 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.258656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.258657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.258664 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.260551 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.260891 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.260893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.260893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.260947 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.261141 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.261143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.261143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.261313 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.261422 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.261564 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.261657 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":53,"threads_started":1}
I20260812 03:48:08.262365 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.252070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58420 (local address 127.7.132.188:44135)
0812 03:48:08.252302 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.252304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.253584 (+ 1280us) server_negotiation.cc:407] Connection header received
0812 03:48:08.255552 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.255553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.255565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.255619 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.256267 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.256916 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.257603 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.257807 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.257879 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.259340 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.259341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.259341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.259348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.260878 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.261258 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.261259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.261260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.261341 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.261567 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.261568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.261568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.261773 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.261876 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.261992 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.262189 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":60,"threads_started":1}
I20260812 03:48:08.262308 17703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:48:08.264258 17703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:08.271276 19656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.267358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:39265 (local address 127.0.0.1:39736)
0812 03:48:08.267538 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.267548 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.267601 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.267885 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.267886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.267888 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:08.267928 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.267930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.269269 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.269270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.270151 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.270154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.271032 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.271033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.271091 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.271095 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.271162 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.271207 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260812 03:48:08.271319 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.267476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39736 (local address 127.7.132.129:39265)
0812 03:48:08.267651 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.267655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.267664 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:08.267701 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.267703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.267725 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.267813 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:08.268009 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.269181 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.270226 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.270949 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.271020 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.271088 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.271177 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.271212 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260812 03:48:08.272150 19663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.267540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:44111 (local address 127.0.0.1:46362)
0812 03:48:08.267900 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.267910 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.267993 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.268286 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.268287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.268288 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:08.268367 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.268368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.269427 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.269428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.270490 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.270493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.271899 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.271901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.271966 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.271969 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.272042 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.272085 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":61,"threads_started":1}
I20260812 03:48:08.272066 19091 tablet_service.cc:1467] Processing CreateTablet for tablet 35dbf6b1df654d13a5cf0c4612c4ee18 (DEFAULT_TABLE table=test-table [id=d23166eaf2bb4b13b20c8ca5a63db8ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:08.272272 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.267888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.7.132.130:45091)
0812 03:48:08.268150 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.268154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.268183 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:48:08.268220 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.268222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.268236 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.268356 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:08.268606 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.270092 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.271254 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.271890 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.271983 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.272074 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.272146 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.272194 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":125,"threads_started":1}
I20260812 03:48:08.272454 19091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35dbf6b1df654d13a5cf0c4612c4ee18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:08.272511 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.267694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46362 (local address 127.7.132.131:44111)
0812 03:48:08.267853 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.267857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.267969 (+ 112us) server_negotiation.cc:407] Connection header received
0812 03:48:08.268105 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.268107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.268123 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.268224 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:08.268441 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.269342 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.270580 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.271280 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.271518 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.271859 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.272311 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.272429 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260812 03:48:08.272410 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.267778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:45091 (local address 127.0.0.1:49126)
0812 03:48:08.267942 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.267950 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.267993 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.268448 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.268449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.268486 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:08.268525 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.268526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.270177 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.270178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.271125 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.271127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.271986 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.271987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.272063 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.272069 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.272131 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.272173 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260812 03:48:08.273224 19516 tablet_service.cc:1467] Processing CreateTablet for tablet 35dbf6b1df654d13a5cf0c4612c4ee18 (DEFAULT_TABLE table=test-table [id=d23166eaf2bb4b13b20c8ca5a63db8ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:08.273553 19516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35dbf6b1df654d13a5cf0c4612c4ee18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:08.276039 19666 tablet_bootstrap.cc:492] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: Bootstrap starting.
I20260812 03:48:08.276482 19307 tablet_service.cc:1467] Processing CreateTablet for tablet 35dbf6b1df654d13a5cf0c4612c4ee18 (DEFAULT_TABLE table=test-table [id=d23166eaf2bb4b13b20c8ca5a63db8ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:08.276634 19666 tablet_bootstrap.cc:654] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:08.276834 19307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35dbf6b1df654d13a5cf0c4612c4ee18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:08.276943 19670 tablet_bootstrap.cc:492] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: Bootstrap starting.
I20260812 03:48:08.277154 19666 log.cc:824] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:08.277573 19670 tablet_bootstrap.cc:654] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:08.277899 19670 log.cc:824] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:08.278050 19666 tablet_bootstrap.cc:492] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: No bootstrap required, opened a new log
I20260812 03:48:08.278149 19666 ts_tablet_manager.cc:1397] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:08.278755 19670 tablet_bootstrap.cc:492] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: No bootstrap required, opened a new log
I20260812 03:48:08.278855 19670 ts_tablet_manager.cc:1397] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:08.280373 19675 tablet_bootstrap.cc:492] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Bootstrap starting.
I20260812 03:48:08.281107 19675 tablet_bootstrap.cc:654] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:08.281191 19670 raft_consensus.cc:348] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.281391 19670 raft_consensus.cc:374] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:08.281428 19675 log.cc:824] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:08.281442 19670 raft_consensus.cc:729] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 038eef5f4c8545f59b60655118824e7e, State: Initialized, Role: FOLLOWER
I20260812 03:48:08.281392 19666 raft_consensus.cc:348] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.281534 19666 raft_consensus.cc:374] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:08.281579 19666 raft_consensus.cc:729] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee97fa4ed1794eb09fdba3945f17ac97, State: Initialized, Role: FOLLOWER
I20260812 03:48:08.281579 19670 consensus_queue.cc:260] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.281700 19666 consensus_queue.cc:260] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.281843 19670 ts_tablet_manager.cc:1428] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:48:08.281893 19624 heartbeater.cc:503] Master 127.7.132.190:40737 was elected leader, sending a full tablet report...
I20260812 03:48:08.281947 19666 ts_tablet_manager.cc:1428] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 03:48:08.282054 19200 heartbeater.cc:503] Master 127.7.132.190:40737 was elected leader, sending a full tablet report...
I20260812 03:48:08.282909 19675 tablet_bootstrap.cc:492] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: No bootstrap required, opened a new log
I20260812 03:48:08.282981 19675 ts_tablet_manager.cc:1397] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:48:08.284998 19675 raft_consensus.cc:348] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.285116 19675 raft_consensus.cc:374] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:08.285137 19675 raft_consensus.cc:729] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18837fec8c8743aab7fe31912772198a, State: Initialized, Role: FOLLOWER
I20260812 03:48:08.285274 19675 consensus_queue.cc:260] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.285518 19675 ts_tablet_manager.cc:1428] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:08.285566 19411 heartbeater.cc:503] Master 127.7.132.190:40737 was elected leader, sending a full tablet report...
I20260812 03:48:08.326602 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.324165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.7.132.188:44135)
0812 03:48:08.324238 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.324240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.324250 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:08.324282 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.324283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.324301 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.324366 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:08.324512 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.325009 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.325892 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.326293 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.326371 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.326437 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.326495 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.326532 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:48:08.326602 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.324088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.0.0.1:58424)
0812 03:48:08.324142 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.324152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.324211 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.324369 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.324370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.324372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:08.324444 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.324446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.325097 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.325098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.325799 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.325801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.326381 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.326382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.326438 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.326443 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.326490 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.326534 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 03:48:08.326982 18927 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260812 03:48:08.327850 19683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2be07f7ce6c84cf286e210ee9e2ac3c1. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:08.327962 19683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: This master's current role is: LEARNER
W20260812 03:48:08.348548 19413 tablet.cc:2369] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:48:08.360208 19680 raft_consensus.cc:482] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:08.360306 19680 raft_consensus.cc:504] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.360586 19680 leader_election.cc:288] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 038eef5f4c8545f59b60655118824e7e (127.7.132.131:44111), 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091)
I20260812 03:48:08.363404 19664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.360880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:46671 (local address 127.7.132.130:45091)
0812 03:48:08.360923 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.360925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.360932 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:48:08.360961 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.360962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.360968 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.361027 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:08.361184 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.361655 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.362612 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.362994 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.363107 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.363116 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.363301 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.363337 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:48:08.363418 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.360743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:45091 (local address 127.7.132.129:46671)
0812 03:48:08.360803 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.360815 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.360868 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.361032 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.361033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.361035 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:08.361099 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.361101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.361740 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.361741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.362500 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.362502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.363104 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.363105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.363192 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.363196 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.363294 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.363337 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":1}
I20260812 03:48:08.363433 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.360745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:44111 (local address 127.7.132.129:36709)
0812 03:48:08.360853 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:08.360863 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:08.360909 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:08.361015 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:08.361016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:08.361017 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:08.361088 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.361090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.361736 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.361737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.362502 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:08.362504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.363145 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.363146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:08.363223 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.363225 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:48:08.363298 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:48:08.363343 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":101}
I20260812 03:48:08.363418 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.360827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:36709 (local address 127.7.132.131:44111)
0812 03:48:08.360868 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.360870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.360895 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:48:08.360958 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.360959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.360969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.361013 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:08.361155 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.361641 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.362608 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.363016 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.363141 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.363149 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.363314 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.363356 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:48:08.363797 19338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35dbf6b1df654d13a5cf0c4612c4ee18" candidate_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18837fec8c8743aab7fe31912772198a" is_pre_election: true
I20260812 03:48:08.363946 19338 raft_consensus.cc:2454] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee97fa4ed1794eb09fdba3945f17ac97 in term 0.
I20260812 03:48:08.363996 19545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35dbf6b1df654d13a5cf0c4612c4ee18" candidate_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "038eef5f4c8545f59b60655118824e7e" is_pre_election: true
I20260812 03:48:08.364123 19545 raft_consensus.cc:2454] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee97fa4ed1794eb09fdba3945f17ac97 in term 0.
I20260812 03:48:08.364166 19023 leader_election.cc:302] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18837fec8c8743aab7fe31912772198a, ee97fa4ed1794eb09fdba3945f17ac97; no voters:
I20260812 03:48:08.364351 19680 raft_consensus.cc:2781] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:08.364401 19680 raft_consensus.cc:482] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:08.364427 19680 raft_consensus.cc:3037] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:08.365178 19680 raft_consensus.cc:504] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.365314 19680 leader_election.cc:288] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [CANDIDATE]: Term 1 election: Requested vote from peers 038eef5f4c8545f59b60655118824e7e (127.7.132.131:44111), 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091)
I20260812 03:48:08.365453 19545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35dbf6b1df654d13a5cf0c4612c4ee18" candidate_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "038eef5f4c8545f59b60655118824e7e"
I20260812 03:48:08.365499 19338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35dbf6b1df654d13a5cf0c4612c4ee18" candidate_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18837fec8c8743aab7fe31912772198a"
I20260812 03:48:08.365559 19338 raft_consensus.cc:3037] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:08.365540 19545 raft_consensus.cc:3037] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:08.366420 19545 raft_consensus.cc:2454] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee97fa4ed1794eb09fdba3945f17ac97 in term 1.
I20260812 03:48:08.366420 19338 raft_consensus.cc:2454] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee97fa4ed1794eb09fdba3945f17ac97 in term 1.
I20260812 03:48:08.366590 19023 leader_election.cc:302] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18837fec8c8743aab7fe31912772198a, ee97fa4ed1794eb09fdba3945f17ac97; no voters:
I20260812 03:48:08.366684 19680 raft_consensus.cc:2781] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:08.366843 19680 raft_consensus.cc:686] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 1 LEADER]: Becoming Leader. State: Replica: ee97fa4ed1794eb09fdba3945f17ac97, State: Running, Role: LEADER
I20260812 03:48:08.366945 19680 consensus_queue.cc:237] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:08.367851 17703 catalog_manager.cc:5495] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee97fa4ed1794eb09fdba3945f17ac97 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } health_report { overall_health: UNKNOWN } } }
I20260812 03:48:08.368502 18860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: NON_VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: true } } }
I20260812 03:48:08.368600 18860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:08.369735 19695 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: attempting to promote NON_VOTER 0a27b760894d4ba892af74b7a05e7ee6 to VOTER
I20260812 03:48:08.369875 19695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } }
I20260812 03:48:08.370289 18634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Refusing update from remote peer 2be07f7ce6c84cf286e210ee9e2ac3c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 03:48:08.370556 18927 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 LEARNER]: Refusing update from remote peer 2be07f7ce6c84cf286e210ee9e2ac3c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 03:48:08.370587 19695 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 03:48:08.370749 19695 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 03:48:08.371171 18860 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:08.371979 18860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:08.372030 19696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:08.372076 18860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:08.372099 19696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:08.372288 18860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:08.372291 18634 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:08.372382 18860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:08.372432 18927 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:08.373225 19702 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:08.373270 19683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:08.373317 19702 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:48:08.373339 19683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:48:08.385294 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.383169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.7.132.129:39265)
0812 03:48:08.383206 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.383208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.383235 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:48:08.383297 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.383298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.383311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.383364 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:08.383474 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.384004 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.384664 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.384836 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.384886 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.385009 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:08.385107 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:08.385164 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.385198 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.385214 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:48:08.386986 19338 raft_consensus.cc:1261] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 1 FOLLOWER]: Refusing update from remote peer ee97fa4ed1794eb09fdba3945f17ac97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:08.386986 19545 raft_consensus.cc:1261] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 1 FOLLOWER]: Refusing update from remote peer ee97fa4ed1794eb09fdba3945f17ac97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:08.387322 19690 consensus_queue.cc:1035] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:08.387444 19680 consensus_queue.cc:1035] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:08.390996 19704 mvcc.cc:205] Tried to move back new op lower bound from 7317530576430460928 to 7317530576351617024. Current Snapshot: MvccSnapshot[applied={T|T < 7317530576430460928 or (T in {7317530576430460928})}]
I20260812 03:48:08.391208 19708 mvcc.cc:205] Tried to move back new op lower bound from 7317530576430460928 to 7317530576351617024. Current Snapshot: MvccSnapshot[applied={T|T < 7317530576430460928}]
I20260812 03:48:08.391942 19707 mvcc.cc:205] Tried to move back new op lower bound from 7317530576430460928 to 7317530576351617024. Current Snapshot: MvccSnapshot[applied={T|T < 7317530576430460928 or (T in {7317530576430460928})}]
I20260812 03:48:08.392783 17703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:08.392870 17703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:08.394157 17703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2be07f7ce6c84cf286e210ee9e2ac3c1: Sending DeleteTablet for 3 replicas of tablet 35dbf6b1df654d13a5cf0c4612c4ee18
I20260812 03:48:08.394482 19091 tablet_service.cc:1514] Processing DeleteTablet for tablet 35dbf6b1df654d13a5cf0c4612c4ee18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:08 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:39736
I20260812 03:48:08.394524 19516 tablet_service.cc:1514] Processing DeleteTablet for tablet 35dbf6b1df654d13a5cf0c4612c4ee18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:08 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:46362
I20260812 03:48:08.394734 19719 tablet_replica.cc:330] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: stopping tablet replica
I20260812 03:48:08.394831 19719 raft_consensus.cc:2229] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:08.394924 19719 raft_consensus.cc:2258] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:08.394948 19307 tablet_service.cc:1514] Processing DeleteTablet for tablet 35dbf6b1df654d13a5cf0c4612c4ee18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:08 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:49126
I20260812 03:48:08.395193 19724 tablet_replica.cc:330] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: stopping tablet replica
I20260812 03:48:08.395279 19724 raft_consensus.cc:2229] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:08.395375 19724 raft_consensus.cc:2258] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:08.395586 19719 ts_tablet_manager.cc:1905] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:08.396039 19721 tablet_replica.cc:330] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: stopping tablet replica
I20260812 03:48:08.396092 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 17648
I20260812 03:48:08.396130 19721 raft_consensus.cc:2229] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:08.408229 19721 raft_consensus.cc:2258] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:08.408407 19724 ts_tablet_manager.cc:1905] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:08.408713 19721 ts_tablet_manager.cc:1905] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:08.410204 19724 ts_tablet_manager.cc:1918] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:08.410274 19724 log.cc:1196] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/35dbf6b1df654d13a5cf0c4612c4ee18
I20260812 03:48:08.410557 19724 ts_tablet_manager.cc:1939] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 18837fec8c8743aab7fe31912772198a: Deleting consensus metadata
I20260812 03:48:08.410650 19721 ts_tablet_manager.cc:1918] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:08.410769 19721 log.cc:1196] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/35dbf6b1df654d13a5cf0c4612c4ee18
I20260812 03:48:08.411054 19721 ts_tablet_manager.cc:1939] T 35dbf6b1df654d13a5cf0c4612c4ee18 P 038eef5f4c8545f59b60655118824e7e: Deleting consensus metadata
W20260812 03:48:08.415062 19226 connection.cc:535] server connection from 127.0.0.1:49126 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 03:48:08.415201 19444 connection.cc:535] server connection from 127.0.0.1:46362 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:48:08.417191 19719 ts_tablet_manager.cc:1918] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:08.417268 19719 log.cc:1196] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/35dbf6b1df654d13a5cf0c4612c4ee18
I20260812 03:48:08.417527 19719 ts_tablet_manager.cc:1939] T 35dbf6b1df654d13a5cf0c4612c4ee18 P ee97fa4ed1794eb09fdba3945f17ac97: Deleting consensus metadata
W20260812 03:48:08.417869 19026 connection.cc:441] server connection from 127.0.0.1:39736 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:39736 (request call id 1) could send its response
I20260812 03:48:08.425303 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.417241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.7.132.189:39453)
0812 03:48:08.417364 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.417366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.417375 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:08.417402 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.417403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.417416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.417478 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.418770 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.419487 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.420290 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.420506 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.420582 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.421981 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.421983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.421983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.421990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.423771 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.424158 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.424159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.424160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.424220 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.424696 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.424698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.424698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.424846 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.424920 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.425119 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.425212 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 03:48:08.425329 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:08.417283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58436 (local address 127.7.132.188:44135)
0812 03:48:08.417330 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:08.417332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:08.417789 (+ 457us) server_negotiation.cc:407] Connection header received
0812 03:48:08.417910 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:08.417911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:08.417922 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:08.417953 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:08.418651 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.419461 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.420567 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:08.420883 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:08.420923 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:08.422354 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:08.422355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:08.422361 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:08.422369 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:08.424214 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.424557 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.424558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.424558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.424603 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:08.424837 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:08.424837 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:08.424838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:08.424943 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:08.425033 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:08.425176 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 03:48:08.425245 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:48:09.392362 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.392055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:58671)
0812 03:48:09.392195 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.392299 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
W20260812 03:48:09.392619 19624 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:40737 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
I20260812 03:48:09.392633 19736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.392384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:57875)
0812 03:48:09.392514 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.392581 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":51,"threads_started":1}
I20260812 03:48:09.392728 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.392419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.130:60237)
0812 03:48:09.392591 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.392679 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
W20260812 03:48:09.392794 19200 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:40737 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
W20260812 03:48:09.392902 19411 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:40737 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
I20260812 03:48:09.895006 19745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2be07f7ce6c84cf286e210ee9e2ac3c1)
I20260812 03:48:09.895002 19744 raft_consensus.cc:482] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2be07f7ce6c84cf286e210ee9e2ac3c1)
I20260812 03:48:09.895128 19745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } }
I20260812 03:48:09.895184 19744 raft_consensus.cc:504] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } }
I20260812 03:48:09.895429 19745 leader_election.cc:288] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737), d528fb1a1c3f46229b4533c6386ed33b (127.7.132.189:39453)
I20260812 03:48:09.895810 19747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.895552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41512)
0812 03:48:09.895746 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.895782 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":63,"threads_started":1}
I20260812 03:48:09.895810 19746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.895553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41504)
0812 03:48:09.895715 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.895771 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":47,"threads_started":1}
W20260812 03:48:09.895915 18892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
W20260812 03:48:09.895916 18598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
I20260812 03:48:09.896009 19744 leader_election.cc:288] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737), 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188:44135)
I20260812 03:48:09.896399 19746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.896317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41532)
0812 03:48:09.896347 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.896374 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260812 03:48:09.896399 19747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.896266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41526)
0812 03:48:09.896357 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.896379 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260812 03:48:09.896474 18892 leader_election.cc:334] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737): Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
W20260812 03:48:09.896553 18598 leader_election.cc:334] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737): Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
I20260812 03:48:09.899473 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.895759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44135 (local address 127.0.0.1:58444)
0812 03:48:09.896098 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.896106 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:09.896173 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:09.896381 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:09.896382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:09.896384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:09.896424 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:09.896425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:09.897283 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:09.897284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:09.898409 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:09.898431 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:09.899192 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:09.899193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:09.899257 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:09.899262 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:48:09.899322 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:48:09.899367 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":59,"threads_started":1}
I20260812 03:48:09.899473 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.895602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:39453 (local address 127.0.0.1:58886)
0812 03:48:09.895739 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.895751 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:09.895808 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:09.896289 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:09.896290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:09.896292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:09.896353 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:09.896355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:09.897283 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:09.897284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:09.898409 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:09.898430 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:09.899192 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:09.899193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:09.899262 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:09.899266 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:09.899322 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:48:09.899367 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":49,"threads_started":1}
I20260812 03:48:09.899475 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.895949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58444 (local address 127.7.132.188:44135)
0812 03:48:09.896254 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:09.896256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:09.896263 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:48:09.896284 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:09.896285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:09.896293 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:09.896351 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:09.896492 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:09.897198 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:09.898531 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:09.899106 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:09.899187 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:09.899196 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:09.899335 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 03:48:09.899376 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":40,"threads_started":1}
I20260812 03:48:09.899485 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.895648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58886 (local address 127.7.132.189:39453)
0812 03:48:09.896142 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:09.896144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:09.896152 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:48:09.896176 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:09.896176 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:09.896185 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:09.896235 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:09.896422 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:09.897191 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:09.898531 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:09.899106 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:09.899184 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:09.899196 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:09.899334 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 03:48:09.899379 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":475,"thread_start_us":40,"threads_started":1}
I20260812 03:48:09.899909 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a27b760894d4ba892af74b7a05e7ee6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d528fb1a1c3f46229b4533c6386ed33b" is_pre_election: true
I20260812 03:48:09.899956 18927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d528fb1a1c3f46229b4533c6386ed33b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0a27b760894d4ba892af74b7a05e7ee6" is_pre_election: true
I20260812 03:48:09.900035 18634 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a27b760894d4ba892af74b7a05e7ee6 in term 1.
I20260812 03:48:09.900053 18927 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d528fb1a1c3f46229b4533c6386ed33b in term 1.
I20260812 03:48:09.900187 18890 leader_election.cc:302] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a27b760894d4ba892af74b7a05e7ee6, d528fb1a1c3f46229b4533c6386ed33b; no voters: 2be07f7ce6c84cf286e210ee9e2ac3c1
I20260812 03:48:09.900197 18597 leader_election.cc:302] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a27b760894d4ba892af74b7a05e7ee6, d528fb1a1c3f46229b4533c6386ed33b; no voters: 2be07f7ce6c84cf286e210ee9e2ac3c1
I20260812 03:48:09.900306 19745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:48:09.900305 19744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:48:09.900347 19744 raft_consensus.cc:482] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2be07f7ce6c84cf286e210ee9e2ac3c1)
I20260812 03:48:09.900345 19745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2be07f7ce6c84cf286e210ee9e2ac3c1)
I20260812 03:48:09.900380 19744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:48:09.900398 19745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:48:09.901065 19745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } }
I20260812 03:48:09.901088 19744 raft_consensus.cc:504] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } }
I20260812 03:48:09.901201 19745 leader_election.cc:288] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 2 election: Requested vote from peers 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737), d528fb1a1c3f46229b4533c6386ed33b (127.7.132.189:39453)
I20260812 03:48:09.901335 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a27b760894d4ba892af74b7a05e7ee6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d528fb1a1c3f46229b4533c6386ed33b"
I20260812 03:48:09.901400 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.901327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41548)
0812 03:48:09.901364 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.901382 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 03:48:09.901389 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.901296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41540)
0812 03:48:09.901335 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.901365 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 03:48:09.901479 18927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d528fb1a1c3f46229b4533c6386ed33b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0a27b760894d4ba892af74b7a05e7ee6"
I20260812 03:48:09.901432 18634 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a27b760894d4ba892af74b7a05e7ee6 in current term 2: Already voted for candidate d528fb1a1c3f46229b4533c6386ed33b in this term.
I20260812 03:48:09.901558 18927 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d528fb1a1c3f46229b4533c6386ed33b in current term 2: Already voted for candidate 0a27b760894d4ba892af74b7a05e7ee6 in this term.
I20260812 03:48:09.901718 19744 leader_election.cc:288] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [CANDIDATE]: Term 2 election: Requested vote from peers 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737), 0a27b760894d4ba892af74b7a05e7ee6 (127.7.132.188:44135)
I20260812 03:48:09.901883 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.901702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41556)
0812 03:48:09.901806 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.901854 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260812 03:48:09.901974 18892 leader_election.cc:334] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737): Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
I20260812 03:48:09.902029 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:09.901918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:41564)
0812 03:48:09.901975 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:09.902008 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:48:09.902042 18892 leader_election.cc:302] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a27b760894d4ba892af74b7a05e7ee6; no voters: 2be07f7ce6c84cf286e210ee9e2ac3c1, d528fb1a1c3f46229b4533c6386ed33b
I20260812 03:48:09.902125 19745 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 03:48:09.902125 18598 leader_election.cc:334] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737): Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
I20260812 03:48:09.902166 18598 leader_election.cc:302] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d528fb1a1c3f46229b4533c6386ed33b; no voters: 0a27b760894d4ba892af74b7a05e7ee6, 2be07f7ce6c84cf286e210ee9e2ac3c1
I20260812 03:48:09.902223 19744 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 03:48:10.393272 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:10.392937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:58965)
0812 03:48:10.393167 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:10.393205 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":114,"threads_started":1}
I20260812 03:48:10.393311 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:10.393098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:51771)
0812 03:48:10.393223 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:10.393258 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
I20260812 03:48:10.393402 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:10.393209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.130:57935)
0812 03:48:10.393323 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:10.393358 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
I20260812 03:48:11.393955 19764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:11.393717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:43817)
0812 03:48:11.393859 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:11.393903 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260812 03:48:11.393980 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:11.393760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.130:35037)
0812 03:48:11.393881 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:11.393927 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260812 03:48:11.394055 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:11.393886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:45013)
0812 03:48:11.393983 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:11.394013 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
I20260812 03:48:11.984974 19774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:11.985100 19774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } }
I20260812 03:48:11.985333 19774 leader_election.cc:288] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737), d528fb1a1c3f46229b4533c6386ed33b (127.7.132.189:39453)
I20260812 03:48:11.985623 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a27b760894d4ba892af74b7a05e7ee6" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d528fb1a1c3f46229b4533c6386ed33b" is_pre_election: true
I20260812 03:48:11.985715 18634 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a27b760894d4ba892af74b7a05e7ee6 in term 2.
I20260812 03:48:11.985684 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:11.985463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:50882)
0812 03:48:11.985581 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:11.985632 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
I20260812 03:48:11.985895 18890 leader_election.cc:302] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a27b760894d4ba892af74b7a05e7ee6, d528fb1a1c3f46229b4533c6386ed33b; no voters:
I20260812 03:48:11.986102 19774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 03:48:11.986159 19774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:11.986196 19774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:48:11.986164 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:11.986070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:50896)
0812 03:48:11.986115 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:11.986143 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 03:48:11.986321 18892 leader_election.cc:334] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737): Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
I20260812 03:48:11.986972 19774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } }
I20260812 03:48:11.987118 19774 leader_election.cc:288] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 3 election: Requested vote from peers 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737), d528fb1a1c3f46229b4533c6386ed33b (127.7.132.189:39453)
I20260812 03:48:11.987295 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:11.987209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:50908)
0812 03:48:11.987244 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:11.987273 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 03:48:11.987308 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a27b760894d4ba892af74b7a05e7ee6" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d528fb1a1c3f46229b4533c6386ed33b"
I20260812 03:48:11.987385 18634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:48:11.987704 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:11.987520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:50918)
0812 03:48:11.987610 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:11.987640 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260812 03:48:11.987795 18892 leader_election.cc:334] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737): Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
I20260812 03:48:11.988091 18634 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a27b760894d4ba892af74b7a05e7ee6 in term 3.
I20260812 03:48:11.988281 18890 leader_election.cc:302] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a27b760894d4ba892af74b7a05e7ee6, d528fb1a1c3f46229b4533c6386ed33b; no voters: 2be07f7ce6c84cf286e210ee9e2ac3c1
I20260812 03:48:11.988391 19774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:48:11.988538 19774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [term 3 LEADER]: Becoming Leader. State: Replica: 0a27b760894d4ba892af74b7a05e7ee6, State: Running, Role: LEADER
I20260812 03:48:11.988641 19774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } }
I20260812 03:48:11.989058 19781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a27b760894d4ba892af74b7a05e7ee6. Latest consensus state: current_term: 3 leader_uuid: "0a27b760894d4ba892af74b7a05e7ee6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:11.989151 19781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:11.989357 19783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:11.989820 19783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:11.989877 19783 catalog_manager.cc:1259] Loaded cluster ID: 00a4a174bbbb4bd0a09cf3a8eab9f15c
I20260812 03:48:11.989893 19783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:11.990027 19783 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:11.990084 19783 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Loaded TSK: 0
I20260812 03:48:11.990185 19783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:48:12.078814 18916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:48:12.080754 18916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:12.081804 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.081670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:50938)
0812 03:48:12.081736 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.081771 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:48:12.081806 18634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [term 3 FOLLOWER]: Refusing update from remote peer 0a27b760894d4ba892af74b7a05e7ee6: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 03:48:12.082027 19781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 03:48:12.082172 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.082052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.0.0.1:50950)
0812 03:48:12.082119 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.082149 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 03:48:12.082239 18892 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 -> Peer 2be07f7ce6c84cf286e210ee9e2ac3c1 (127.7.132.190:40737): Couldn't send request to peer 2be07f7ce6c84cf286e210ee9e2ac3c1. Status: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:48:12.082692 19785 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a27b760894d4ba892af74b7a05e7ee6. Latest consensus state: current_term: 3 leader_uuid: "0a27b760894d4ba892af74b7a05e7ee6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:12.082842 19785 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:48:12.083196 19784 mvcc.cc:205] Tried to move back new op lower bound from 7317530591565037568 to 7317530591186194432. Current Snapshot: MvccSnapshot[applied={T|T < 7317530591565037568}]
I20260812 03:48:12.083328 19781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "0a27b760894d4ba892af74b7a05e7ee6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:12.083420 19781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:12.083434 19785 sys_catalog.cc:455] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "0a27b760894d4ba892af74b7a05e7ee6" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "2be07f7ce6c84cf286e210ee9e2ac3c1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40737 } } peers { permanent_uuid: "d528fb1a1c3f46229b4533c6386ed33b" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 39453 } attrs { promote: false } } peers { permanent_uuid: "0a27b760894d4ba892af74b7a05e7ee6" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44135 } attrs { promote: false } } }
I20260812 03:48:12.083537 19785 sys_catalog.cc:458] T 00000000000000000000000000000000 P d528fb1a1c3f46229b4533c6386ed33b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:48:12.083601 19787 mvcc.cc:205] Tried to move back new op lower bound from 7317530591565037568 to 7317530591186194432. Current Snapshot: MvccSnapshot[applied={T|T < 7317530591565037568}]
I20260812 03:48:12.088490 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.085264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:45091 (local address 127.0.0.1:50108)
0812 03:48:12.085467 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.085476 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:12.085520 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:12.085922 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:12.085924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:12.085925 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:12.085957 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.085958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.086652 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.086653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.087513 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.087516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.088236 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.088238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.088298 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.088301 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:12.088372 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:48:12.088414 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":73,"threads_started":1}
I20260812 03:48:12.088502 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.085607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50108 (local address 127.7.132.130:45091)
0812 03:48:12.085745 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:12.085747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:12.085757 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:12.085784 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:12.085784 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:12.085795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:12.085866 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:12.086040 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.086592 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.087608 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.088153 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.088234 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.088247 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:12.088391 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:48:12.088433 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260812 03:48:12.088784 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.085340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:39265 (local address 127.0.0.1:45826)
0812 03:48:12.085750 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.085761 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:12.085806 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:12.086018 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:12.086019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:12.086021 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:12.086081 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.086083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.086991 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.086992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.087827 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.087830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.088572 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.088573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.088632 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.088636 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:12.088678 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 03:48:12.088712 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":79,"threads_started":1}
I20260812 03:48:12.089011 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.085398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45826 (local address 127.7.132.129:39265)
0812 03:48:12.085501 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:12.085503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:12.085858 (+ 355us) server_negotiation.cc:407] Connection header received
0812 03:48:12.085892 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:12.085893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:12.085906 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:12.085966 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:12.086165 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.086915 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.087899 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.088465 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.088529 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.088539 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:12.088876 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 03:48:12.088922 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":59,"threads_started":1}
I20260812 03:48:12.089108 19307 tablet_service.cc:1467] Processing CreateTablet for tablet b8f3c8bed3a1491bb02dc949fe6d7732 (DEFAULT_TABLE table=test-table [id=4f22a4e96dbf4325824463774967a644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:12.089255 19307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8f3c8bed3a1491bb02dc949fe6d7732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:12.089890 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.085333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.7.132.131:44111)
0812 03:48:12.085647 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:12.085649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:12.085658 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:12.085687 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:12.085688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:12.085701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:12.086052 (+ 351us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:12.086098 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.087889 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.088999 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.089547 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.089605 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.089613 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:12.089780 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 03:48:12.089811 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":77,"threads_started":1}
I20260812 03:48:12.089902 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.085230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:44111 (local address 127.0.0.1:53400)
0812 03:48:12.085453 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.085464 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:12.085520 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:12.085802 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:12.085803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:12.085805 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:12.085864 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.085866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.087954 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.087954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.088931 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.088934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.089628 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.089629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.089685 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.089689 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:12.089752 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:48:12.089803 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210}
I20260812 03:48:12.090754 19803 tablet_bootstrap.cc:492] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: Bootstrap starting.
I20260812 03:48:12.090770 19091 tablet_service.cc:1467] Processing CreateTablet for tablet b8f3c8bed3a1491bb02dc949fe6d7732 (DEFAULT_TABLE table=test-table [id=4f22a4e96dbf4325824463774967a644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:12.090907 19091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8f3c8bed3a1491bb02dc949fe6d7732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:12.091156 19803 tablet_bootstrap.cc:654] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:12.091848 19803 tablet_bootstrap.cc:492] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: No bootstrap required, opened a new log
I20260812 03:48:12.091912 19803 ts_tablet_manager.cc:1397] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:12.092051 19803 raft_consensus.cc:348] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.092125 19803 raft_consensus.cc:374] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:12.092165 19803 raft_consensus.cc:729] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18837fec8c8743aab7fe31912772198a, State: Initialized, Role: FOLLOWER
I20260812 03:48:12.092219 19803 consensus_queue.cc:260] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.092375 19803 ts_tablet_manager.cc:1428] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:12.092428 19516 tablet_service.cc:1467] Processing CreateTablet for tablet b8f3c8bed3a1491bb02dc949fe6d7732 (DEFAULT_TABLE table=test-table [id=4f22a4e96dbf4325824463774967a644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:12.092484 19805 tablet_bootstrap.cc:492] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97: Bootstrap starting.
I20260812 03:48:12.092545 19516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8f3c8bed3a1491bb02dc949fe6d7732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:12.092757 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.092564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.130:39373)
0812 03:48:12.092684 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.092720 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":83,"threads_started":1}
I20260812 03:48:12.092846 19805 tablet_bootstrap.cc:654] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:12.093468 19805 tablet_bootstrap.cc:492] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97: No bootstrap required, opened a new log
I20260812 03:48:12.093531 19805 ts_tablet_manager.cc:1397] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:12.093662 19805 raft_consensus.cc:348] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.093734 19805 raft_consensus.cc:374] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:12.093753 19805 raft_consensus.cc:729] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee97fa4ed1794eb09fdba3945f17ac97, State: Initialized, Role: FOLLOWER
I20260812 03:48:12.093812 19805 consensus_queue.cc:260] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.094153 19819 tablet_bootstrap.cc:492] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: Bootstrap starting.
I20260812 03:48:12.094699 19819 tablet_bootstrap.cc:654] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:12.093950 19805 ts_tablet_manager.cc:1428] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:12.094823 19198 heartbeater.cc:503] Master 127.7.132.188:44135 was elected leader, sending a full tablet report...
I20260812 03:48:12.095027 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.094150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:32869)
0812 03:48:12.094960 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.094998 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":93,"threads_started":1}
I20260812 03:48:12.095396 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.095265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:35729)
0812 03:48:12.095340 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.095371 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 03:48:12.095695 19819 tablet_bootstrap.cc:492] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: No bootstrap required, opened a new log
I20260812 03:48:12.095767 19819 ts_tablet_manager.cc:1397] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:48:12.095913 19819 raft_consensus.cc:348] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.095999 19819 raft_consensus.cc:374] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:12.096020 19819 raft_consensus.cc:729] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 038eef5f4c8545f59b60655118824e7e, State: Initialized, Role: FOLLOWER
I20260812 03:48:12.096086 19819 consensus_queue.cc:260] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.096287 19819 ts_tablet_manager.cc:1428] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:12.096676 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.096456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:55469)
0812 03:48:12.096595 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.096632 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":96,"threads_started":1}
I20260812 03:48:12.096846 19623 heartbeater.cc:503] Master 127.7.132.188:44135 was elected leader, sending a full tablet report...
I20260812 03:48:12.097002 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.096933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:34933)
0812 03:48:12.096962 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.096984 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260812 03:48:12.167523 19824 raft_consensus.cc:482] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:12.167624 19824 raft_consensus.cc:504] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.168030 19824 leader_election.cc:288] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee97fa4ed1794eb09fdba3945f17ac97 (127.7.132.129:39265), 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091)
I20260812 03:48:12.170928 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.168213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:39265 (local address 127.7.132.131:38721)
0812 03:48:12.168312 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.168324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:12.168414 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:12.168589 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:12.168590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:12.168593 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:12.168651 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.168653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.169332 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.169333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.170082 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.170084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.170647 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.170648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.170759 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.170763 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:12.170812 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:48:12.170852 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 03:48:12.170928 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.168306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:38721 (local address 127.7.132.129:39265)
0812 03:48:12.168375 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:12.168377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:12.168386 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:12.168502 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:12.168504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:12.168519 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:12.168576 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:12.168731 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.169240 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.170201 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.170561 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.170646 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.170664 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:12.170832 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 03:48:12.170869 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:48:12.171069 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.168301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:39587 (local address 127.7.132.130:45091)
0812 03:48:12.168372 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:12.168374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:12.168643 (+ 269us) server_negotiation.cc:407] Connection header received
0812 03:48:12.168718 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:12.168719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:12.168737 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:12.168806 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:12.168946 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.169422 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.170308 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.170661 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.170814 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.170826 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:12.170971 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:48:12.171008 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:48:12.171085 19826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.168213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:45091 (local address 127.7.132.131:39587)
0812 03:48:12.168566 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.168575 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:12.168653 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:12.168811 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:12.168812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:12.168814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:12.168865 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.168867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.169512 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.169513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.170205 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:12.170207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.170825 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.170826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:12.170892 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.170895 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:12.170963 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:48:12.171001 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":123,"thread_start_us":161,"threads_started":1}
I20260812 03:48:12.171324 19122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8f3c8bed3a1491bb02dc949fe6d7732" candidate_uuid: "038eef5f4c8545f59b60655118824e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" is_pre_election: true
I20260812 03:48:12.171375 19338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8f3c8bed3a1491bb02dc949fe6d7732" candidate_uuid: "038eef5f4c8545f59b60655118824e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18837fec8c8743aab7fe31912772198a" is_pre_election: true
I20260812 03:48:12.171461 19122 raft_consensus.cc:2454] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 038eef5f4c8545f59b60655118824e7e in term 0.
I20260812 03:48:12.171492 19338 raft_consensus.cc:2454] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 038eef5f4c8545f59b60655118824e7e in term 0.
I20260812 03:48:12.171674 19445 leader_election.cc:302] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 038eef5f4c8545f59b60655118824e7e, ee97fa4ed1794eb09fdba3945f17ac97; no voters:
I20260812 03:48:12.171818 19824 raft_consensus.cc:2781] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:12.171857 19824 raft_consensus.cc:482] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:12.171886 19824 raft_consensus.cc:3037] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:12.172564 19824 raft_consensus.cc:504] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.172705 19824 leader_election.cc:288] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [CANDIDATE]: Term 1 election: Requested vote from peers ee97fa4ed1794eb09fdba3945f17ac97 (127.7.132.129:39265), 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091)
I20260812 03:48:12.172883 19122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8f3c8bed3a1491bb02dc949fe6d7732" candidate_uuid: "038eef5f4c8545f59b60655118824e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee97fa4ed1794eb09fdba3945f17ac97"
I20260812 03:48:12.172889 19338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8f3c8bed3a1491bb02dc949fe6d7732" candidate_uuid: "038eef5f4c8545f59b60655118824e7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18837fec8c8743aab7fe31912772198a"
I20260812 03:48:12.172959 19122 raft_consensus.cc:3037] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:12.172959 19338 raft_consensus.cc:3037] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:12.173682 19338 raft_consensus.cc:2454] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 038eef5f4c8545f59b60655118824e7e in term 1.
I20260812 03:48:12.173817 19122 raft_consensus.cc:2454] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 038eef5f4c8545f59b60655118824e7e in term 1.
I20260812 03:48:12.173885 19440 leader_election.cc:302] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 038eef5f4c8545f59b60655118824e7e, 18837fec8c8743aab7fe31912772198a; no voters:
I20260812 03:48:12.174009 19824 raft_consensus.cc:2781] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:12.174187 19824 raft_consensus.cc:686] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 1 LEADER]: Becoming Leader. State: Replica: 038eef5f4c8545f59b60655118824e7e, State: Running, Role: LEADER
I20260812 03:48:12.174374 19824 consensus_queue.cc:237] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } }
I20260812 03:48:12.174522 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.174427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:55859)
0812 03:48:12.174462 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.174497 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 03:48:12.175145 18913 catalog_manager.cc:5495] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 038eef5f4c8545f59b60655118824e7e (127.7.132.131). New cstate: current_term: 1 leader_uuid: "038eef5f4c8545f59b60655118824e7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "038eef5f4c8545f59b60655118824e7e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 } health_report { overall_health: UNKNOWN } } }
I20260812 03:48:12.181852 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.179616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.7.132.131:44111)
0812 03:48:12.179675 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:12.179677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:12.179709 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:48:12.179805 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:12.179810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:12.179826 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:12.179903 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:12.180022 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.180457 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.181276 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:12.181451 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:12.181504 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:12.181620 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:12.181706 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:12.181755 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:12.181792 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 03:48:12.181807 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:48:12.182823 19338 raft_consensus.cc:1261] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 1 FOLLOWER]: Refusing update from remote peer 038eef5f4c8545f59b60655118824e7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:12.182981 19122 raft_consensus.cc:1261] T b8f3c8bed3a1491bb02dc949fe6d7732 P ee97fa4ed1794eb09fdba3945f17ac97 [term 1 FOLLOWER]: Refusing update from remote peer 038eef5f4c8545f59b60655118824e7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:12.183071 19824 consensus_queue.cc:1035] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "18837fec8c8743aab7fe31912772198a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 45091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:12.183267 19824 consensus_queue.cc:1035] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee97fa4ed1794eb09fdba3945f17ac97" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:12.183495 19410 heartbeater.cc:503] Master 127.7.132.188:44135 was elected leader, sending a full tablet report...
I20260812 03:48:12.183770 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.183656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.130:44533)
0812 03:48:12.183702 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.183739 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:48:12.184237 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.184130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:59539)
0812 03:48:12.184173 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.184202 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 03:48:12.185343 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.185026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:49973)
0812 03:48:12.185288 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.185319 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246}
I20260812 03:48:12.185693 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.185605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.130:33989)
0812 03:48:12.185642 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.185666 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 03:48:12.186028 19825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.185944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.131:35609)
0812 03:48:12.185978 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.186002 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 03:48:12.186786 19821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.186678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40737 (local address 127.7.132.129:48041)
0812 03:48:12.186735 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.186761 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:40737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 03:48:12.187171 18913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:12.187269 18913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:12.188611 18913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a27b760894d4ba892af74b7a05e7ee6: Sending DeleteTablet for 3 replicas of tablet b8f3c8bed3a1491bb02dc949fe6d7732
I20260812 03:48:12.188905 19516 tablet_service.cc:1514] Processing DeleteTablet for tablet b8f3c8bed3a1491bb02dc949fe6d7732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:12 UTC) from {username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:53400
I20260812 03:48:12.188923 19091 tablet_service.cc:1514] Processing DeleteTablet for tablet b8f3c8bed3a1491bb02dc949fe6d7732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:12 UTC) from {username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:45826
I20260812 03:48:12.189152 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 18991
I20260812 03:48:12.201265 19307 tablet_service.cc:1514] Processing DeleteTablet for tablet b8f3c8bed3a1491bb02dc949fe6d7732 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:12 UTC) from {username='oryx', principal='oryx/127.7.132.188@KRBTEST.COM'} at 127.0.0.1:50108
I20260812 03:48:12.201378 19852 tablet_replica.cc:330] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: stopping tablet replica
I20260812 03:48:12.201450 19852 raft_consensus.cc:2229] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:12.201542 19852 raft_consensus.cc:2258] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:12.201571 19854 tablet_replica.cc:330] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: stopping tablet replica
I20260812 03:48:12.201639 19854 raft_consensus.cc:2229] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:12.201691 19854 raft_consensus.cc:2258] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:12.202023 19854 ts_tablet_manager.cc:1905] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:12.202359 19852 ts_tablet_manager.cc:1905] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:12.203655 19854 ts_tablet_manager.cc:1918] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:12.203725 19854 log.cc:1196] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/b8f3c8bed3a1491bb02dc949fe6d7732
I20260812 03:48:12.203963 19854 ts_tablet_manager.cc:1939] T b8f3c8bed3a1491bb02dc949fe6d7732 P 18837fec8c8743aab7fe31912772198a: Deleting consensus metadata
I20260812 03:48:12.204303 18890 catalog_manager.cc:4841] TS 18837fec8c8743aab7fe31912772198a (127.7.132.130:45091): tablet b8f3c8bed3a1491bb02dc949fe6d7732 (table test-table [id=4f22a4e96dbf4325824463774967a644]) successfully deleted
I20260812 03:48:12.204629 19852 ts_tablet_manager.cc:1918] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:12.204703 19852 log.cc:1196] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/b8f3c8bed3a1491bb02dc949fe6d7732
I20260812 03:48:12.204964 19852 ts_tablet_manager.cc:1939] T b8f3c8bed3a1491bb02dc949fe6d7732 P 038eef5f4c8545f59b60655118824e7e: Deleting consensus metadata
I20260812 03:48:12.205340 18894 catalog_manager.cc:4841] TS 038eef5f4c8545f59b60655118824e7e (127.7.132.131:44111): tablet b8f3c8bed3a1491bb02dc949fe6d7732 (table test-table [id=4f22a4e96dbf4325824463774967a644]) successfully deleted
I20260812 03:48:12.209204 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:12.209096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:39265 (local address 127.0.0.1:45832)
0812 03:48:12.209139 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:12.209170 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:39265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 03:48:12.209365 18894 catalog_manager.cc:4584] TS ee97fa4ed1794eb09fdba3945f17ac97 (127.7.132.129:39265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8f3c8bed3a1491bb02dc949fe6d7732: Network error: Client connection negotiation failed: client connection to 127.7.132.129:39265: connect: Connection refused (error 111)
I20260812 03:48:12.209438 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 19206
I20260812 03:48:12.232447 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 19420
I20260812 03:48:12.244899 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 18405
I20260812 03:48:12.258831 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 18705
2026-08-12T03:48:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7030 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:48:12 dist-test-slave-39l8 krb5kdc[19859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:48:12 dist-test-slave-39l8 krb5kdc[19859](info): set up 2 sockets
Aug 12 03:48:12 dist-test-slave-39l8 krb5kdc[19859](info): commencing operation
krb5kdc: starting...
W20260812 03:48:13.489812 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:48:13 dist-test-slave-39l8 krb5kdc[19859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:48:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.190.keytab.
Entry for principal oryx/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.190.keytab.
I20260812 03:48:13.605080 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46293
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.190.keytab
--principal=oryx/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:13.684943 19883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:13.685181 19883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:13.685230 19883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:13.686334 19883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:13.686398 19883 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:48:13.686419 19883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:13.686442 19883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:13.686486 19883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:13.686512 19883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:13.687980 19883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46293
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.190.keytab
--principal=oryx/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:13.688241 19883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:13.688446 19883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:13.693426 19889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:13.693461 19890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:13.693462 19883 server_base.cc:1034] running on GCE node
W20260812 03:48:13.693440 19893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:13.693905 19883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:13.694199 19883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:13.695408 19883 hybrid_clock.cc:648] HybridClock initialized: now 1786506493695384 us; error 39 us; skew 500 ppm
Aug 12 03:48:13 dist-test-slave-39l8 krb5kdc[19859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506493, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:13.697918 19883 init.cc:377] Logged in from keytab as oryx/127.7.132.190@KRBTEST.COM (short username oryx)
I20260812 03:48:13.698354 19883 webserver.cc:460] Webserver started at http://127.7.132.190:35751/ using document root <none> and password file <none>
I20260812 03:48:13.698596 19883 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:13.698673 19883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:13.698882 19883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:13.699959 19883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "b109da4fdd0344038ce94f8cf1753d6f"
format_stamp: "Formatted at 2026-08-12 03:48:13 on dist-test-slave-39l8"
server_key: "5f7122f316a001be522761cd71fbe6a8"
server_key_iv: "ecaae3b015082e01edb7694d31f747b8"
server_key_version: "encryptionkey@0"
I20260812 03:48:13.700323 19883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "b109da4fdd0344038ce94f8cf1753d6f"
format_stamp: "Formatted at 2026-08-12 03:48:13 on dist-test-slave-39l8"
server_key: "5f7122f316a001be522761cd71fbe6a8"
server_key_iv: "ecaae3b015082e01edb7694d31f747b8"
server_key_version: "encryptionkey@0"
I20260812 03:48:13.701663 19883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:13.702406 19906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:13.702581 19883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:13.702692 19883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "b109da4fdd0344038ce94f8cf1753d6f"
format_stamp: "Formatted at 2026-08-12 03:48:13 on dist-test-slave-39l8"
server_key: "5f7122f316a001be522761cd71fbe6a8"
server_key_iv: "ecaae3b015082e01edb7694d31f747b8"
server_key_version: "encryptionkey@0"
I20260812 03:48:13.702806 19883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:13.715129 19883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:13.716770 19883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:13.716922 19883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:13.721644 19883 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46293
I20260812 03:48:13.721714 20014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46293 every 8 connection(s)
I20260812 03:48:13.722087 19883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:13.722882 20016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:13.725114 20016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: Bootstrap starting.
I20260812 03:48:13.725724 20016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:13.726022 20016 log.cc:824] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:13.726670 20016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: No bootstrap required, opened a new log
I20260812 03:48:13.728701 20016 raft_consensus.cc:348] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b109da4fdd0344038ce94f8cf1753d6f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46293 } }
I20260812 03:48:13.728838 20016 raft_consensus.cc:374] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:13.728861 20016 raft_consensus.cc:729] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b109da4fdd0344038ce94f8cf1753d6f, State: Initialized, Role: FOLLOWER
I20260812 03:48:13.729019 20016 consensus_queue.cc:260] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b109da4fdd0344038ce94f8cf1753d6f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46293 } }
I20260812 03:48:13.729095 20016 raft_consensus.cc:388] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:13.729117 20016 raft_consensus.cc:482] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:13.729148 20016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:13.729593 20016 raft_consensus.cc:504] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b109da4fdd0344038ce94f8cf1753d6f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46293 } }
I20260812 03:48:13.729674 20016 leader_election.cc:302] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b109da4fdd0344038ce94f8cf1753d6f; no voters:
I20260812 03:48:13.729804 20016 leader_election.cc:288] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:13.729857 20019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:13.730050 20019 raft_consensus.cc:686] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [term 1 LEADER]: Becoming Leader. State: Replica: b109da4fdd0344038ce94f8cf1753d6f, State: Running, Role: LEADER
I20260812 03:48:13.730079 20016 sys_catalog.cc:564] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:13.730244 20019 consensus_queue.cc:237] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b109da4fdd0344038ce94f8cf1753d6f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46293 } }
I20260812 03:48:13.730351 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 19883
I20260812 03:48:13.730460 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:13.730563 20019 sys_catalog.cc:455] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b109da4fdd0344038ce94f8cf1753d6f. Latest consensus state: current_term: 1 leader_uuid: "b109da4fdd0344038ce94f8cf1753d6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b109da4fdd0344038ce94f8cf1753d6f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46293 } } }
I20260812 03:48:13.730651 20019 sys_catalog.cc:458] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:13.730697 7698 external_mini_cluster.cc:1428] Setting key 755b08d93c8a2b94780d4be75bd1cc82
I20260812 03:48:13.730896 20030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:13.731428 20030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:13.731829 20023 sys_catalog.cc:455] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b109da4fdd0344038ce94f8cf1753d6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b109da4fdd0344038ce94f8cf1753d6f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46293 } } }
I20260812 03:48:13.731964 20023 sys_catalog.cc:458] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:13.733515 20030 catalog_manager.cc:1347] Generated new cluster ID: 1e8422c6478a4774a4533a8cbce2e979
I20260812 03:48:13.733569 20030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:48:13 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:13.742079 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:13.731313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46650 (local address 127.7.132.190:46293)
0812 03:48:13.731557 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:13.731567 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:13.731606 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:48:13.731731 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:13.731745 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:13.731824 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:13.731968 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:13.732689 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.734099 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.735116 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.735719 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.735853 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:13.738550 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:13.738554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:13.738554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:13.738566 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:13.740457 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:13.740859 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:13.740862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:13.740862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:13.740960 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:13.741176 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:13.741176 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:13.741177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:13.741449 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:13.741603 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:13.741722 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 03:48:13.741866 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":72,"threads_started":1}
I20260812 03:48:13.744171 20030 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:13.744808 20030 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:13.748745 20030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: Generated new TSK 0
I20260812 03:48:13.748919 20030 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
I20260812 03:48:13.775592 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46293
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:13.856292 20058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:13.856516 20058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:13.856556 20058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:13.857643 20058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:13.857707 20058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:13.857738 20058 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:48:13.857772 20058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:13.859324 20058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:46293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:13.859576 20058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:13.859813 20058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:13.864969 20066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:13.864982 20072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:13.865016 20058 server_base.cc:1034] running on GCE node
W20260812 03:48:13.864984 20067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:13.865393 20058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:13.865657 20058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:13.866770 20058 hybrid_clock.cc:648] HybridClock initialized: now 1786506493866750 us; error 34 us; skew 500 ppm
Aug 12 03:48:13 dist-test-slave-39l8 krb5kdc[19859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506493, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:13.869210 20058 init.cc:377] Logged in from keytab as oryx/127.7.132.129@KRBTEST.COM (short username oryx)
I20260812 03:48:13.869681 20058 webserver.cc:460] Webserver started at http://127.7.132.129:43845/ using document root <none> and password file <none>
I20260812 03:48:13.869961 20058 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:13.870033 20058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:13.870193 20058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:13.871338 20058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "6644672d0cc245b5b24724c1289b3ddb"
format_stamp: "Formatted at 2026-08-12 03:48:13 on dist-test-slave-39l8"
server_key: "300b0c3ebde90041f4f053d50e7275f7"
server_key_iv: "0664ff726d0e32ecc52cdf150daae680"
server_key_version: "encryptionkey@0"
I20260812 03:48:13.871711 20058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "6644672d0cc245b5b24724c1289b3ddb"
format_stamp: "Formatted at 2026-08-12 03:48:13 on dist-test-slave-39l8"
server_key: "300b0c3ebde90041f4f053d50e7275f7"
server_key_iv: "0664ff726d0e32ecc52cdf150daae680"
server_key_version: "encryptionkey@0"
I20260812 03:48:13.873046 20058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:48:13.873735 20090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:13.873903 20058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:13.874003 20058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "6644672d0cc245b5b24724c1289b3ddb"
format_stamp: "Formatted at 2026-08-12 03:48:13 on dist-test-slave-39l8"
server_key: "300b0c3ebde90041f4f053d50e7275f7"
server_key_iv: "0664ff726d0e32ecc52cdf150daae680"
server_key_version: "encryptionkey@0"
I20260812 03:48:13.874078 20058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:13.894507 20058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:13.896497 20058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:13.896656 20058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:13.897274 20058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:13.897331 20058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:13.897367 20058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:13.897393 20058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:48:13 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506493, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:13.904724 20058 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:41569
I20260812 03:48:13.904819 20281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:41569 every 8 connection(s)
I20260812 03:48:13.905207 20058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:48:13.907259 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:13.897663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:49199 (local address 127.7.132.190:46293)
0812 03:48:13.897732 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:13.897735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:13.897940 (+ 205us) server_negotiation.cc:407] Connection header received
0812 03:48:13.898440 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:13.898441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:13.898458 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:13.898547 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:13.899269 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.899809 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.900810 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.901020 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.901118 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:13.904226 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:13.904227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:13.904227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:13.904239 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:13.905998 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:13.906468 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:13.906468 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:13.906469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:13.906519 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:13.906696 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:13.906696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:13.906696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:13.906840 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:13.906889 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:13.907128 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 03:48:13.907192 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:48:13.907414 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:13.897653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46293 (local address 127.7.132.129:49199)
0812 03:48:13.897810 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:13.897849 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:13.898319 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:13.898618 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:13.898623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:13.898926 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:13.899157 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:13.899163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.899908 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.899909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:13.900692 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:13.900694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.901130 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.901131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:13.901373 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:13.901915 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:13.901923 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:13.904111 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:13.906069 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:13.906074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:13.906076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:13.906397 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:13.906562 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:13.906563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:13.906563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:13.906660 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:13.906898 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:13.906904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:13.907013 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 03:48:13.907099 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:48:13.907216 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260812 03:48:13.907897 20282 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46293
I20260812 03:48:13.907979 20282 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:13.908178 20282 heartbeater.cc:511] Master 127.7.132.190:46293 requested a full tablet report, sending...
I20260812 03:48:13.908759 19930 ts_manager.cc:194] Registered new tserver with Master: 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569)
I20260812 03:48:13.909586 19930 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:49199
I20260812 03:48:13.911530 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 20058
I20260812 03:48:13.911667 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:13.911900 7698 external_mini_cluster.cc:1428] Setting key 1a21261497c32a6bdeda79ff24585fdd
I20260812 03:48:13.914418 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:13.908909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46293 (local address 127.7.132.129:44259)
0812 03:48:13.909050 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:13.909078 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:13.909126 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:13.909239 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:13.909239 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:13.909377 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:13.909429 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:13.909430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.910139 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.910140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:13.910869 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:13.910872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.911180 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.911181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:13.911227 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:13.911672 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:13.911677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:13.911974 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:13.913575 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:13.913576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:13.913576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:13.913737 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:13.913921 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:13.913922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:13.913922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:13.913962 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:13.914180 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:13.914181 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:13.914226 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:48:13.914285 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:48:13.914354 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":33,"thread_start_us":89,"threads_started":1}
I20260812 03:48:13.914659 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:13.908962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:44259 (local address 127.7.132.190:46293)
0812 03:48:13.908999 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:13.909001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:13.909142 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:48:13.909164 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:13.909165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:13.909174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:13.909236 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:13.909488 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.910049 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.910951 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:13.911116 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:13.911168 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:13.912050 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:13.912051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:13.912052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:13.912059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:13.913494 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:13.913806 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:13.913807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:13.913808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:13.913865 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:13.913998 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:13.913999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:13.913999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:13.914119 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:13.914180 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:13.914332 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:48:13.914414 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for oryx/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
I20260812 03:48:13.935428 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46293
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:14.017102 20295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:14.017277 20295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:14.017297 20295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:14.019002 20295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:14.019059 20295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:14.019080 20295 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:48:14.019121 20295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:14.020677 20295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:46293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:14.020936 20295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:14.021180 20295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:14.026055 20301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:14.026087 20295 server_base.cc:1034] running on GCE node
W20260812 03:48:14.026108 20304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:14.026059 20306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:14.026466 20295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:14.026760 20295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:14.027926 20295 hybrid_clock.cc:648] HybridClock initialized: now 1786506494027924 us; error 39 us; skew 500 ppm
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506494, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:14.030463 20295 init.cc:377] Logged in from keytab as oryx/127.7.132.130@KRBTEST.COM (short username oryx)
I20260812 03:48:14.031009 20295 webserver.cc:460] Webserver started at http://127.7.132.130:41379/ using document root <none> and password file <none>
I20260812 03:48:14.031260 20295 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:14.031303 20295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:14.031451 20295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:14.032513 20295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "03a6a2c621bc43ebaad7400db9a3072c"
format_stamp: "Formatted at 2026-08-12 03:48:14 on dist-test-slave-39l8"
server_key: "facfc0ede2fcff7407dc6d11a09f10af"
server_key_iv: "f81dbcb6e66667784d51f20e116e9bcf"
server_key_version: "encryptionkey@0"
I20260812 03:48:14.032864 20295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "03a6a2c621bc43ebaad7400db9a3072c"
format_stamp: "Formatted at 2026-08-12 03:48:14 on dist-test-slave-39l8"
server_key: "facfc0ede2fcff7407dc6d11a09f10af"
server_key_iv: "f81dbcb6e66667784d51f20e116e9bcf"
server_key_version: "encryptionkey@0"
I20260812 03:48:14.034144 20295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:48:14.034876 20315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.035065 20295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.035161 20295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "03a6a2c621bc43ebaad7400db9a3072c"
format_stamp: "Formatted at 2026-08-12 03:48:14 on dist-test-slave-39l8"
server_key: "facfc0ede2fcff7407dc6d11a09f10af"
server_key_iv: "f81dbcb6e66667784d51f20e116e9bcf"
server_key_version: "encryptionkey@0"
I20260812 03:48:14.035228 20295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:14.044510 20295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:14.046201 20295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:14.046350 20295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:14.046862 20295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:14.046902 20295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.046922 20295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:14.046934 20295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.053867 20295 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:40933
I20260812 03:48:14.053948 20497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:40933 every 8 connection(s)
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506494, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:14.054301 20295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:14.057580 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.047310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:42349 (local address 127.7.132.190:46293)
0812 03:48:14.047397 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.047400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.047506 (+ 106us) server_negotiation.cc:407] Connection header received
0812 03:48:14.047971 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.047973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.047985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.048035 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.048681 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.049441 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.051098 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.051412 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.051484 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.054833 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:14.054834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:14.054835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:14.054842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:14.056226 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.056628 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.056630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.056630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.056690 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.056937 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.056938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.056938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.057047 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:14.057124 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.057382 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.057466 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:48:14.057777 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.047260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46293 (local address 127.7.132.130:42349)
0812 03:48:14.047400 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.047435 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.047863 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.048094 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.048098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.048332 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:14.048592 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.048597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.049552 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.049553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.050985 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.050989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.051513 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.051514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.051678 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.052812 (+ 1134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:14.052823 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:14.054690 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:14.056332 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.056340 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.056342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.056553 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.056758 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.056759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.056760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.056868 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.057116 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:14.057122 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:14.057254 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.057369 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.057549 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260812 03:48:14.058282 20498 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46293
I20260812 03:48:14.058383 20498 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:14.058547 20498 heartbeater.cc:511] Master 127.7.132.190:46293 requested a full tablet report, sending...
I20260812 03:48:14.058940 19929 ts_manager.cc:194] Registered new tserver with Master: 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933)
I20260812 03:48:14.059442 19929 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:42349
I20260812 03:48:14.060098 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 20295
I20260812 03:48:14.060217 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:48:14.060427 7698 external_mini_cluster.cc:1428] Setting key d0e5eac7c8d6d55e2df6473b8ab53a85
I20260812 03:48:14.065475 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.059317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:33129 (local address 127.7.132.190:46293)
0812 03:48:14.059359 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.059361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.059416 (+ 55us) server_negotiation.cc:407] Connection header received
0812 03:48:14.059476 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.059476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.059484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.059555 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.059903 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.060620 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.061296 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.061543 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.061611 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.062593 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:14.062594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:14.062594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:14.062600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:14.064353 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.064696 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.064697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.064697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.064746 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.065001 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.065002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.065002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.065133 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:14.065187 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.065370 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.065423 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:48:14.065593 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.059223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46293 (local address 127.7.132.130:33129)
0812 03:48:14.059354 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.059364 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.059423 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.059559 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.059560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.059750 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:14.059809 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.059810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.060721 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.060722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.061207 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.061208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.061631 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.061632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.061722 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.062111 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:14.062117 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:14.062507 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:14.064451 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.064452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.064452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.064616 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.064866 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.064867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.064867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.064936 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.065205 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:14.065206 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:14.065257 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.065344 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.065454 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for oryx/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
I20260812 03:48:14.086417 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46293
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:14.167927 20513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:14.168147 20513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:14.168185 20513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:14.169306 20513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:14.169371 20513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:14.169405 20513 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:48:14.169446 20513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:14.171010 20513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.132.190:46293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:14.171280 20513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:14.171511 20513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:14.176319 20524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:14.176295 20522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:14.176290 20521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:14.176618 20513 server_base.cc:1034] running on GCE node
I20260812 03:48:14.176878 20513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:14.177116 20513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:14.178262 20513 hybrid_clock.cc:648] HybridClock initialized: now 1786506494178250 us; error 30 us; skew 500 ppm
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506494, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:14.180688 20513 init.cc:377] Logged in from keytab as oryx/127.7.132.131@KRBTEST.COM (short username oryx)
I20260812 03:48:14.181164 20513 webserver.cc:460] Webserver started at http://127.7.132.131:41541/ using document root <none> and password file <none>
I20260812 03:48:14.181421 20513 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:14.181489 20513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:14.181640 20513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:14.182667 20513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "9c0ed608e33d4650842ba5a8f7388a31"
format_stamp: "Formatted at 2026-08-12 03:48:14 on dist-test-slave-39l8"
server_key: "003ab0d5dd3441e2ded01dd163a3abd0"
server_key_iv: "ac73441d643dfc1a46aa14907b444c5e"
server_key_version: "encryptionkey@0"
I20260812 03:48:14.183094 20513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "9c0ed608e33d4650842ba5a8f7388a31"
format_stamp: "Formatted at 2026-08-12 03:48:14 on dist-test-slave-39l8"
server_key: "003ab0d5dd3441e2ded01dd163a3abd0"
server_key_iv: "ac73441d643dfc1a46aa14907b444c5e"
server_key_version: "encryptionkey@0"
I20260812 03:48:14.184427 20513 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.185115 20537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.185302 20513 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:14.185410 20513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "9c0ed608e33d4650842ba5a8f7388a31"
format_stamp: "Formatted at 2026-08-12 03:48:14 on dist-test-slave-39l8"
server_key: "003ab0d5dd3441e2ded01dd163a3abd0"
server_key_iv: "ac73441d643dfc1a46aa14907b444c5e"
server_key_version: "encryptionkey@0"
I20260812 03:48:14.185484 20513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:14.193807 20513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:14.195668 20513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:14.195827 20513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:14.196369 20513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:14.196444 20513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.196477 20513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:14.196518 20513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506494, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:14.203857 20513 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:33649
I20260812 03:48:14.203944 20723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:33649 every 8 connection(s)
I20260812 03:48:14.204263 20513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:14.206044 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.197117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:46163 (local address 127.7.132.190:46293)
0812 03:48:14.197186 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.197188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.197199 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:48:14.197618 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.197619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.197632 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.197695 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.198351 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.199009 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.200030 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.200254 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.200331 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.203090 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:14.203092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:14.203092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:14.203101 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:14.204864 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.205267 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.205268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.205268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.205334 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.205529 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.205530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.205530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.205631 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:14.205686 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.205898 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.205958 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:48:14.206149 20550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.196770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46293 (local address 127.7.132.131:46163)
0812 03:48:14.196934 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.196971 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.197515 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.197776 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.197779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.198070 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:14.198249 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.198254 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.199101 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.199102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.199900 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.199902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.200339 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.200340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.200468 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.200955 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:14.200962 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:14.202977 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:14.204964 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.204968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.204969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.205172 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.205394 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.205394 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.205394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.205474 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.205679 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:14.205683 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:14.205790 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.205866 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.205976 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260812 03:48:14.206611 20726 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46293
I20260812 03:48:14.206732 20726 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:14.206907 20726 heartbeater.cc:511] Master 127.7.132.190:46293 requested a full tablet report, sending...
I20260812 03:48:14.207300 19929 ts_manager.cc:194] Registered new tserver with Master: 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649)
I20260812 03:48:14.207814 19929 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:46163
I20260812 03:48:14.211424 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 20513
I20260812 03:48:14.211552 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:14.211795 7698 external_mini_cluster.cc:1428] Setting key 2a109afff71e6bc8f4fa37fb498981fa
I20260812 03:48:14.212730 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:48:14.213030 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.207630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:52809 (local address 127.7.132.190:46293)
0812 03:48:14.207676 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.207678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.207745 (+ 67us) server_negotiation.cc:407] Connection header received
0812 03:48:14.207805 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.207806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.207813 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.207856 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.208239 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.208687 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.209483 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.209684 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.209770 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.210561 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:14.210562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:14.210563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:14.210570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:14.211910 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.212283 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.212284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.212285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.212338 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.212577 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.212579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.212579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.212701 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:14.212758 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.212910 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.212975 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:48:14.213055 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.207547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46293 (local address 127.7.132.131:52809)
0812 03:48:14.207686 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.207696 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.207752 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.207861 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.207862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.208094 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:14.208149 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.208151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.208784 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.208785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.209387 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.209389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.209774 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.209775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.209848 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.210216 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:14.210220 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:14.210472 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:14.211990 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.211991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.211991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.212213 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.212411 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.212412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.212412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.212490 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.212754 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:14.212755 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:14.212805 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.212873 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.212965 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506494, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:48:14.235878 20044 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:14.229419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.7.132.190:46293)
0812 03:48:14.229481 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.229482 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.229504 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:48:14.229576 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.229577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.229586 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.229635 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.230259 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.230887 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.231723 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.231903 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.231955 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.235785 (+ 3830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46666: BlockingRecv error: recv got EOF from 127.0.0.1:46666 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506494, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:14.242904 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.236768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.7.132.190:46293)
0812 03:48:14.236810 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.236812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.236853 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:48:14.236901 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.236902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.236909 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.236954 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.237368 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.237872 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.238497 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.238739 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.238808 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.240811 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:14.240812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:14.240813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:14.240819 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:14.241951 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.242263 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.242264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.242264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.242314 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.242466 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.242466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.242467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.242551 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:14.242621 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.242697 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.242815 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:48:14.244307 19929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:48:14.245838 19929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:14.252453 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.248502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.7.132.130:40933)
0812 03:48:14.249015 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.249020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.249029 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:14.249066 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.249068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.249081 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.249166 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.249340 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.250049 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.251489 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.251998 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.252112 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.252212 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.252319 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.252362 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":102,"threads_started":1}
I20260812 03:48:14.252444 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.248033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:40933 (local address 127.0.0.1:43318)
0812 03:48:14.248491 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.248502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.248559 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.249189 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.249192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.249194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.249259 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.249261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.250137 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.250138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.251423 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.251426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.252118 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.252119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.252212 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.252229 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.252303 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.252357 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":22,"thread_start_us":52,"threads_started":1}
I20260812 03:48:14.252667 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.248230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33649 (local address 127.0.0.1:39202)
0812 03:48:14.248996 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.249005 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.249061 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.249293 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.249294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.249296 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.249351 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.249353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.250208 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.250209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.251238 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.251240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.252283 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.252284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.252343 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.252347 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.252392 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.252433 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":53,"thread_start_us":40,"threads_started":1}
I20260812 03:48:14.253165 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.248521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39202 (local address 127.7.132.131:33649)
0812 03:48:14.248775 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.248782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.249056 (+ 274us) server_negotiation.cc:407] Connection header received
0812 03:48:14.249134 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.249137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.249154 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.249226 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.249412 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.250117 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.251331 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.252100 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.252171 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.252248 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.252495 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.253071 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":121,"threads_started":1}
I20260812 03:48:14.253227 20393 tablet_service.cc:1467] Processing CreateTablet for tablet ce4aba0595e84143933168be40f22c60 (DEFAULT_TABLE table=test-table [id=1b8480d887a64e1694b10b3d281248bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.253628 20393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4aba0595e84143933168be40f22c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.253788 20618 tablet_service.cc:1467] Processing CreateTablet for tablet ce4aba0595e84143933168be40f22c60 (DEFAULT_TABLE table=test-table [id=1b8480d887a64e1694b10b3d281248bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.254093 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4aba0595e84143933168be40f22c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.255678 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.248030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41569 (local address 127.0.0.1:35988)
0812 03:48:14.248144 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.248220 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.248317 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.249240 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.249242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.249244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.249341 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.249343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.251394 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.251395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.252439 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.252441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.255426 (+ 2985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.255428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.255503 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.255507 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.255575 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.255611 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":22,"thread_start_us":55,"threads_started":1}
I20260812 03:48:14.255766 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.248304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.7.132.129:41569)
0812 03:48:14.248730 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.248735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.248746 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:48:14.248786 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.248788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.248805 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.248894 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.250241 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.251315 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.252527 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.255324 (+ 2797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.255407 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.255485 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.255599 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.255643 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":80,"threads_started":1}
I20260812 03:48:14.256692 20183 tablet_service.cc:1467] Processing CreateTablet for tablet ce4aba0595e84143933168be40f22c60 (DEFAULT_TABLE table=test-table [id=1b8480d887a64e1694b10b3d281248bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.257040 20183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4aba0595e84143933168be40f22c60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.257292 20770 tablet_bootstrap.cc:492] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Bootstrap starting.
I20260812 03:48:14.257514 20771 tablet_bootstrap.cc:492] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Bootstrap starting.
I20260812 03:48:14.258121 20771 tablet_bootstrap.cc:654] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.258121 20770 tablet_bootstrap.cc:654] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.258488 20771 log.cc:824] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:14.258488 20770 log.cc:824] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:14.259892 20771 tablet_bootstrap.cc:492] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: No bootstrap required, opened a new log
I20260812 03:48:14.259905 20770 tablet_bootstrap.cc:492] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: No bootstrap required, opened a new log
I20260812 03:48:14.259970 20771 ts_tablet_manager.cc:1397] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:48:14.259970 20770 ts_tablet_manager.cc:1397] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:48:14.260572 20775 tablet_bootstrap.cc:492] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Bootstrap starting.
I20260812 03:48:14.261149 20775 tablet_bootstrap.cc:654] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.261439 20775 log.cc:824] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:14.261970 20771 raft_consensus.cc:348] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.261970 20770 raft_consensus.cc:348] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.262144 20771 raft_consensus.cc:374] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.262144 20770 raft_consensus.cc:374] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.262190 20770 raft_consensus.cc:729] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c0ed608e33d4650842ba5a8f7388a31, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.262187 20771 raft_consensus.cc:729] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a6a2c621bc43ebaad7400db9a3072c, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.262193 20775 tablet_bootstrap.cc:492] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: No bootstrap required, opened a new log
I20260812 03:48:14.262257 20775 ts_tablet_manager.cc:1397] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:48:14.262373 20771 consensus_queue.cc:260] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.262310 20770 consensus_queue.cc:260] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.262676 20771 ts_tablet_manager.cc:1428] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:48:14.262666 20770 ts_tablet_manager.cc:1428] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:48:14.262812 20498 heartbeater.cc:503] Master 127.7.132.190:46293 was elected leader, sending a full tablet report...
I20260812 03:48:14.263128 20726 heartbeater.cc:503] Master 127.7.132.190:46293 was elected leader, sending a full tablet report...
I20260812 03:48:14.264276 20775 raft_consensus.cc:348] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.264397 20775 raft_consensus.cc:374] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.264420 20775 raft_consensus.cc:729] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6644672d0cc245b5b24724c1289b3ddb, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.264523 20775 consensus_queue.cc:260] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.264752 20775 ts_tablet_manager.cc:1428] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:14.264827 20282 heartbeater.cc:503] Master 127.7.132.190:46293 was elected leader, sending a full tablet report...
W20260812 03:48:14.304944 20499 tablet.cc:2369] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:48:14.406131 20283 tablet.cc:2369] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:48:14.454949 20728 tablet.cc:2369] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:48:14.484375 20779 raft_consensus.cc:482] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:14.484493 20779 raft_consensus.cc:504] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.484891 20779 leader_election.cc:288] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933), 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649)
I20260812 03:48:14.487776 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.485165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:54011 (local address 127.7.132.130:40933)
0812 03:48:14.485224 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.485226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.485239 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:48:14.485327 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.485328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.485339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.485404 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.485587 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.486053 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.487081 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.487399 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.487518 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.487527 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.487670 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.487706 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:48:14.487793 20780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.485062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:40933 (local address 127.7.132.129:54011)
0812 03:48:14.485181 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.485193 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.485256 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.485420 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.485422 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.485425 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.485513 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.485515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.486138 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.486139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.486998 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.487001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.487523 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.487524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.487594 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.487597 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.487662 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.487700 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":2,"thread_start_us":76,"threads_started":1}
I20260812 03:48:14.487836 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.485354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:36635 (local address 127.7.132.131:33649)
0812 03:48:14.485429 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.485432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.485442 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:14.485488 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.485489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.485498 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.485568 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.485735 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.486167 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.487083 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.487482 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.487556 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.487564 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.487709 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.487751 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:48:14.487897 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.485248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33649 (local address 127.7.132.129:36635)
0812 03:48:14.485367 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.485392 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.485436 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.485572 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.485573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.485575 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.485656 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.485658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.486249 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.486250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.487000 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.487002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.487566 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.487567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.487634 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.487637 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.487701 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.487741 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260812 03:48:14.488258 20641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4aba0595e84143933168be40f22c60" candidate_uuid: "6644672d0cc245b5b24724c1289b3ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c0ed608e33d4650842ba5a8f7388a31" is_pre_election: true
I20260812 03:48:14.488268 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4aba0595e84143933168be40f22c60" candidate_uuid: "6644672d0cc245b5b24724c1289b3ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a6a2c621bc43ebaad7400db9a3072c" is_pre_election: true
I20260812 03:48:14.488410 20641 raft_consensus.cc:2454] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6644672d0cc245b5b24724c1289b3ddb in term 0.
I20260812 03:48:14.488410 20430 raft_consensus.cc:2454] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6644672d0cc245b5b24724c1289b3ddb in term 0.
I20260812 03:48:14.488632 20096 leader_election.cc:302] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a6a2c621bc43ebaad7400db9a3072c, 6644672d0cc245b5b24724c1289b3ddb; no voters:
I20260812 03:48:14.488822 20779 raft_consensus.cc:2781] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:14.488909 20779 raft_consensus.cc:482] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:14.488940 20779 raft_consensus.cc:3037] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.489645 20779 raft_consensus.cc:504] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.489771 20779 leader_election.cc:288] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [CANDIDATE]: Term 1 election: Requested vote from peers 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933), 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649)
I20260812 03:48:14.489933 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4aba0595e84143933168be40f22c60" candidate_uuid: "6644672d0cc245b5b24724c1289b3ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a6a2c621bc43ebaad7400db9a3072c"
I20260812 03:48:14.489962 20641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4aba0595e84143933168be40f22c60" candidate_uuid: "6644672d0cc245b5b24724c1289b3ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c0ed608e33d4650842ba5a8f7388a31"
I20260812 03:48:14.490013 20430 raft_consensus.cc:3037] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.490026 20641 raft_consensus.cc:3037] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.490897 20430 raft_consensus.cc:2454] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6644672d0cc245b5b24724c1289b3ddb in term 1.
I20260812 03:48:14.490897 20641 raft_consensus.cc:2454] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6644672d0cc245b5b24724c1289b3ddb in term 1.
I20260812 03:48:14.491060 20096 leader_election.cc:302] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6644672d0cc245b5b24724c1289b3ddb, 9c0ed608e33d4650842ba5a8f7388a31; no voters:
I20260812 03:48:14.491181 20779 raft_consensus.cc:2781] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:14.491294 20779 raft_consensus.cc:686] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 1 LEADER]: Becoming Leader. State: Replica: 6644672d0cc245b5b24724c1289b3ddb, State: Running, Role: LEADER
I20260812 03:48:14.491400 20779 consensus_queue.cc:237] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } }
I20260812 03:48:14.492213 19927 catalog_manager.cc:5495] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb reported cstate change: term changed from 0 to 1, leader changed from <none> to 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129). New cstate: current_term: 1 leader_uuid: "6644672d0cc245b5b24724c1289b3ddb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } health_report { overall_health: HEALTHY } } }
I20260812 03:48:14.536008 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.533535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36000 (local address 127.7.132.129:41569)
0812 03:48:14.533593 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.533595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.533604 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:14.533645 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.533646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.533666 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.533751 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:14.533915 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.534412 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.535258 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.535456 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.535527 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.535695 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.535799 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.535888 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.535911 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.535931 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:48:14.537837 20641 raft_consensus.cc:1261] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Refusing update from remote peer 6644672d0cc245b5b24724c1289b3ddb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:14.537904 20430 raft_consensus.cc:1261] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Refusing update from remote peer 6644672d0cc245b5b24724c1289b3ddb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:14.538220 20779 consensus_queue.cc:1035] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:14.538403 20786 consensus_queue.cc:1035] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:14.540947 20789 mvcc.cc:205] Tried to move back new op lower bound from 7317530601624313856 to 7317530601437433856. Current Snapshot: MvccSnapshot[applied={T|T < 7317530601624313856}]
I20260812 03:48:14.541036 20790 mvcc.cc:205] Tried to move back new op lower bound from 7317530601624313856 to 7317530601437433856. Current Snapshot: MvccSnapshot[applied={T|T < 7317530601624313856}]
I20260812 03:48:14.541042 20791 mvcc.cc:205] Tried to move back new op lower bound from 7317530601624313856 to 7317530601437433856. Current Snapshot: MvccSnapshot[applied={T|T < 7317530601624313856}]
I20260812 03:48:14.543375 19930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:14.543465 19930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:14.544255 19930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: Sending DeleteTablet for 3 replicas of tablet ce4aba0595e84143933168be40f22c60
I20260812 03:48:14.544597 20183 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4aba0595e84143933168be40f22c60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:35988
I20260812 03:48:14.544613 20393 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4aba0595e84143933168be40f22c60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43318
I20260812 03:48:14.544601 20618 tablet_service.cc:1514] Processing DeleteTablet for tablet ce4aba0595e84143933168be40f22c60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:39202
I20260812 03:48:14.544862 20805 tablet_replica.cc:330] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: stopping tablet replica
I20260812 03:48:14.544986 20805 raft_consensus.cc:2229] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:14.544968 19930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:48:14.545095 20805 raft_consensus.cc:2258] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:48:14.545104 19930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:14.545424 20804 tablet_replica.cc:330] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: stopping tablet replica
I20260812 03:48:14.545567 20804 raft_consensus.cc:2229] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:14.545742 20804 raft_consensus.cc:2258] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:14.546475 20804 ts_tablet_manager.cc:1905] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.546482 20618 tablet_service.cc:1467] Processing CreateTablet for tablet c6644b9694ae4ca189b4fd09f4262b57 (DEFAULT_TABLE table=test-table [id=a21e2b1b55ac462893bc5ba666c7d826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.546685 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6644b9694ae4ca189b4fd09f4262b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.546697 20183 tablet_service.cc:1467] Processing CreateTablet for tablet c6644b9694ae4ca189b4fd09f4262b57 (DEFAULT_TABLE table=test-table [id=a21e2b1b55ac462893bc5ba666c7d826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.546823 20183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6644b9694ae4ca189b4fd09f4262b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.547009 20805 ts_tablet_manager.cc:1905] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.548170 20770 tablet_bootstrap.cc:492] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: Bootstrap starting.
I20260812 03:48:14.548200 20775 tablet_bootstrap.cc:492] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: Bootstrap starting.
I20260812 03:48:14.548583 20805 ts_tablet_manager.cc:1918] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:14.548609 20770 tablet_bootstrap.cc:654] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.548609 20775 tablet_bootstrap.cc:654] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.548656 20805 log.cc:1196] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/ce4aba0595e84143933168be40f22c60
I20260812 03:48:14.548919 20805 ts_tablet_manager.cc:1939] T ce4aba0595e84143933168be40f22c60 P 6644672d0cc245b5b24724c1289b3ddb: Deleting consensus metadata
I20260812 03:48:14.549023 20804 ts_tablet_manager.cc:1918] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:14.549101 20804 log.cc:1196] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/ce4aba0595e84143933168be40f22c60
I20260812 03:48:14.549351 19907 catalog_manager.cc:4841] TS 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569): tablet ce4aba0595e84143933168be40f22c60 (table test-table [id=1b8480d887a64e1694b10b3d281248bb]) successfully deleted
I20260812 03:48:14.549392 20804 ts_tablet_manager.cc:1939] T ce4aba0595e84143933168be40f22c60 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting consensus metadata
I20260812 03:48:14.549573 20775 tablet_bootstrap.cc:492] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: No bootstrap required, opened a new log
I20260812 03:48:14.549618 20775 ts_tablet_manager.cc:1397] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.549674 20770 tablet_bootstrap.cc:492] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: No bootstrap required, opened a new log
I20260812 03:48:14.549715 20770 ts_tablet_manager.cc:1397] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:48:14.549782 20775 raft_consensus.cc:348] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.549829 20775 raft_consensus.cc:374] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.549841 20775 raft_consensus.cc:729] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6644672d0cc245b5b24724c1289b3ddb, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.549865 20770 raft_consensus.cc:348] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.549898 20775 consensus_queue.cc:260] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.549922 20770 raft_consensus.cc:374] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.549943 20770 raft_consensus.cc:729] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c0ed608e33d4650842ba5a8f7388a31, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.549973 20770 consensus_queue.cc:260] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.549983 20775 ts_tablet_manager.cc:1428] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.550048 20770 ts_tablet_manager.cc:1428] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.550107 20393 tablet_service.cc:1467] Processing CreateTablet for tablet c6644b9694ae4ca189b4fd09f4262b57 (DEFAULT_TABLE table=test-table [id=a21e2b1b55ac462893bc5ba666c7d826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.550222 20393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6644b9694ae4ca189b4fd09f4262b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.550222 19909 catalog_manager.cc:4841] TS 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649): tablet ce4aba0595e84143933168be40f22c60 (table test-table [id=1b8480d887a64e1694b10b3d281248bb]) successfully deleted
I20260812 03:48:14.551121 20806 tablet_replica.cc:330] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: stopping tablet replica
I20260812 03:48:14.551262 20806 raft_consensus.cc:2229] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:14.551399 20806 raft_consensus.cc:2258] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:14.551582 20771 tablet_bootstrap.cc:492] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: Bootstrap starting.
I20260812 03:48:14.551970 20806 ts_tablet_manager.cc:1905] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.552063 20771 tablet_bootstrap.cc:654] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.552661 20771 tablet_bootstrap.cc:492] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: No bootstrap required, opened a new log
I20260812 03:48:14.552726 20771 ts_tablet_manager.cc:1397] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.552882 20771 raft_consensus.cc:348] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.552963 20771 raft_consensus.cc:374] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.552991 20771 raft_consensus.cc:729] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a6a2c621bc43ebaad7400db9a3072c, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.553035 20771 consensus_queue.cc:260] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.553141 20771 ts_tablet_manager.cc:1428] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:14.553411 20806 ts_tablet_manager.cc:1918] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:14.553479 20806 log.cc:1196] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/ce4aba0595e84143933168be40f22c60
I20260812 03:48:14.553745 20806 ts_tablet_manager.cc:1939] T ce4aba0595e84143933168be40f22c60 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting consensus metadata
I20260812 03:48:14.554174 19909 catalog_manager.cc:4841] TS 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933): tablet ce4aba0595e84143933168be40f22c60 (table test-table [id=1b8480d887a64e1694b10b3d281248bb]) successfully deleted
I20260812 03:48:14.642676 20777 raft_consensus.cc:482] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:14.642822 20777 raft_consensus.cc:504] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.643153 20777 leader_election.cc:288] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569), 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649)
I20260812 03:48:14.646366 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.643336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41569 (local address 127.7.132.130:46797)
0812 03:48:14.643560 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.643570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.643638 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.643837 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.643837 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.643839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.643897 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.643898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.644826 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.644827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.645561 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.645563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.646110 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.646111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.646185 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.646189 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.646249 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.646293 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260812 03:48:14.646363 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.643458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:46797 (local address 127.7.132.129:41569)
0812 03:48:14.643508 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.643510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.643646 (+ 136us) server_negotiation.cc:407] Connection header received
0812 03:48:14.643705 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.643706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.643717 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.643752 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.643975 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.644712 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.645654 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.645990 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.646106 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.646116 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.646268 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.646301 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:48:14.646806 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6644b9694ae4ca189b4fd09f4262b57" candidate_uuid: "03a6a2c621bc43ebaad7400db9a3072c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6644672d0cc245b5b24724c1289b3ddb" is_pre_election: true
I20260812 03:48:14.646945 20191 raft_consensus.cc:2454] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a6a2c621bc43ebaad7400db9a3072c in term 0.
I20260812 03:48:14.647454 20316 leader_election.cc:302] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a6a2c621bc43ebaad7400db9a3072c, 6644672d0cc245b5b24724c1289b3ddb; no voters:
I20260812 03:48:14.647614 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.643469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:53729 (local address 127.7.132.131:33649)
0812 03:48:14.643512 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.643514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.643678 (+ 164us) server_negotiation.cc:407] Connection header received
0812 03:48:14.643706 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.643707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.643717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.643793 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.644874 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.645591 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.646826 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.647230 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.647310 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.647316 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.647483 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.647544 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:48:14.647607 20777 raft_consensus.cc:2781] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:14.647707 20777 raft_consensus.cc:482] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:14.647732 20777 raft_consensus.cc:3037] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.647867 20812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.643326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33649 (local address 127.7.132.130:53729)
0812 03:48:14.643558 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.643570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.643638 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.643814 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.643816 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.643818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.643896 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.643898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.645688 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.645690 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.646735 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.646739 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.647334 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.647347 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.647416 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.647420 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.647477 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.647530 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":18,"thread_start_us":60,"threads_started":1}
I20260812 03:48:14.648173 20641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6644b9694ae4ca189b4fd09f4262b57" candidate_uuid: "03a6a2c621bc43ebaad7400db9a3072c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c0ed608e33d4650842ba5a8f7388a31" is_pre_election: true
I20260812 03:48:14.648273 20641 raft_consensus.cc:2454] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03a6a2c621bc43ebaad7400db9a3072c in term 0.
I20260812 03:48:14.648680 20777 raft_consensus.cc:504] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.648830 20777 leader_election.cc:288] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [CANDIDATE]: Term 1 election: Requested vote from peers 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569), 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649)
I20260812 03:48:14.648981 20641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6644b9694ae4ca189b4fd09f4262b57" candidate_uuid: "03a6a2c621bc43ebaad7400db9a3072c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c0ed608e33d4650842ba5a8f7388a31"
I20260812 03:48:14.649089 20641 raft_consensus.cc:3037] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.649101 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6644b9694ae4ca189b4fd09f4262b57" candidate_uuid: "03a6a2c621bc43ebaad7400db9a3072c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6644672d0cc245b5b24724c1289b3ddb"
I20260812 03:48:14.649184 20191 raft_consensus.cc:3037] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.649786 20191 raft_consensus.cc:2454] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a6a2c621bc43ebaad7400db9a3072c in term 1.
I20260812 03:48:14.649844 20641 raft_consensus.cc:2454] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03a6a2c621bc43ebaad7400db9a3072c in term 1.
I20260812 03:48:14.649979 20316 leader_election.cc:302] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a6a2c621bc43ebaad7400db9a3072c, 6644672d0cc245b5b24724c1289b3ddb; no voters:
I20260812 03:48:14.650084 20777 raft_consensus.cc:2781] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:14.650205 20777 raft_consensus.cc:686] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 LEADER]: Becoming Leader. State: Replica: 03a6a2c621bc43ebaad7400db9a3072c, State: Running, Role: LEADER
I20260812 03:48:14.650297 20777 consensus_queue.cc:237] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.651010 19930 catalog_manager.cc:5495] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c reported cstate change: term changed from 0 to 1, leader changed from <none> to 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130). New cstate: current_term: 1 leader_uuid: "03a6a2c621bc43ebaad7400db9a3072c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506493, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.190@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:14.690129 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.683442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46293 (local address 127.0.0.1:46688)
0812 03:48:14.683619 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.683630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.683702 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.683838 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.683839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.684180 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:14.684236 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.684239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.684900 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.684901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.685481 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.685482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.685801 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.685802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.685842 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.686330 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:14.686334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:14.687771 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:14.689222 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.689236 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.689238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.689425 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.689595 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:14.689596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:14.689596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:14.689653 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:14.689856 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:14.689862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:14.689901 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.689953 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.690038 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260812 03:48:14.690181 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.683504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.7.132.190:46293)
0812 03:48:14.683575 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.683578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.683703 (+ 125us) server_negotiation.cc:407] Connection header received
0812 03:48:14.683761 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.683762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.683772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.683813 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.684308 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.684830 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.685555 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.685733 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.685786 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.687859 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:14.687860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:14.687860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:14.687866 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:14.689108 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.689487 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.689487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.689488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.689537 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.689705 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.689705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.689705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.689801 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:14.689852 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.689983 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.690085 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":43}
I20260812 03:48:14.692219 19930 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:46688:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:48:14.692466 19930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:14.693940 20183 tablet_service.cc:1467] Processing CreateTablet for tablet 7764783aaa104d2484146ede82c5e82d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f7c1edacd4624a93b5b21eb896c8b7ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:48:14.693938 20393 tablet_service.cc:1467] Processing CreateTablet for tablet 7764783aaa104d2484146ede82c5e82d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f7c1edacd4624a93b5b21eb896c8b7ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:48:14.693938 20618 tablet_service.cc:1467] Processing CreateTablet for tablet 7764783aaa104d2484146ede82c5e82d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f7c1edacd4624a93b5b21eb896c8b7ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:48:14.694101 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7764783aaa104d2484146ede82c5e82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.694101 20393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7764783aaa104d2484146ede82c5e82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.694116 20183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7764783aaa104d2484146ede82c5e82d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.695307 20775 tablet_bootstrap.cc:492] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: Bootstrap starting.
I20260812 03:48:14.695556 20771 tablet_bootstrap.cc:492] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: Bootstrap starting.
I20260812 03:48:14.695564 20770 tablet_bootstrap.cc:492] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: Bootstrap starting.
I20260812 03:48:14.695657 20775 tablet_bootstrap.cc:654] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.695966 20771 tablet_bootstrap.cc:654] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.695994 20770 tablet_bootstrap.cc:654] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.696244 20775 tablet_bootstrap.cc:492] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: No bootstrap required, opened a new log
I20260812 03:48:14.696295 20775 ts_tablet_manager.cc:1397] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.696434 20775 raft_consensus.cc:348] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.696514 20775 raft_consensus.cc:374] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.696540 20775 raft_consensus.cc:729] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6644672d0cc245b5b24724c1289b3ddb, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.696587 20775 consensus_queue.cc:260] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.696655 20771 tablet_bootstrap.cc:492] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: No bootstrap required, opened a new log
I20260812 03:48:14.696681 20775 ts_tablet_manager.cc:1428] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.696698 20771 ts_tablet_manager.cc:1397] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.696738 20770 tablet_bootstrap.cc:492] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: No bootstrap required, opened a new log
I20260812 03:48:14.696713 20794 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.696782 20770 ts_tablet_manager.cc:1397] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.696781 20794 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:48:14.696848 20771 raft_consensus.cc:348] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.696914 20771 raft_consensus.cc:374] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.696936 20771 raft_consensus.cc:729] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a6a2c621bc43ebaad7400db9a3072c, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.696924 20770 raft_consensus.cc:348] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.696986 20770 raft_consensus.cc:374] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.697007 20770 raft_consensus.cc:729] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c0ed608e33d4650842ba5a8f7388a31, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.696985 20771 consensus_queue.cc:260] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.697045 20770 consensus_queue.cc:260] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.697075 20771 ts_tablet_manager.cc:1428] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.697134 20770 ts_tablet_manager.cc:1428] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:14.697161 20777 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.697263 20777 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:48:14.697256 20778 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.697319 20778 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:48:14.743418 20778 raft_consensus.cc:482] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:14.743506 20778 raft_consensus.cc:504] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.743873 20778 leader_election.cc:288] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569), 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933)
I20260812 03:48:14.746675 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.744125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49397 (local address 127.7.132.130:40933)
0812 03:48:14.744175 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.744177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.744263 (+ 86us) server_negotiation.cc:407] Connection header received
0812 03:48:14.744334 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.744335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.744344 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.744396 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.744540 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.745013 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.745983 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.746336 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.746414 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.746423 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.746573 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.746606 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:48:14.746755 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.744048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:40933 (local address 127.7.132.131:49397)
0812 03:48:14.744182 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.744191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.744282 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.744400 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.744401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.744402 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.744471 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.744475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.745096 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.745097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.745898 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.745900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.746413 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.746414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.746478 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.746482 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.746567 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.746605 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":30,"thread_start_us":51,"threads_started":1}
I20260812 03:48:14.746742 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.744093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:42207 (local address 127.7.132.129:41569)
0812 03:48:14.744157 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.744159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.744253 (+ 94us) server_negotiation.cc:407] Connection header received
0812 03:48:14.744341 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.744342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.744350 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.744412 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:14.744561 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.745012 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.745990 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.746351 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.746437 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.746445 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.746610 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.746655 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:48:14.746974 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.744012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41569 (local address 127.7.132.131:42207)
0812 03:48:14.744178 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.744189 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.744271 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.744415 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.744416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.744418 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:14.744489 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.744491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.745102 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.745103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.745899 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.745902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.746437 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.746438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.746509 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.746512 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.746601 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.746641 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":31,"thread_start_us":79,"threads_started":1}
I20260812 03:48:14.747072 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7764783aaa104d2484146ede82c5e82d" candidate_uuid: "9c0ed608e33d4650842ba5a8f7388a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a6a2c621bc43ebaad7400db9a3072c" is_pre_election: true
I20260812 03:48:14.747197 20430 raft_consensus.cc:2454] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c0ed608e33d4650842ba5a8f7388a31 in term 0.
I20260812 03:48:14.747370 20540 leader_election.cc:302] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a6a2c621bc43ebaad7400db9a3072c, 9c0ed608e33d4650842ba5a8f7388a31; no voters:
I20260812 03:48:14.747373 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7764783aaa104d2484146ede82c5e82d" candidate_uuid: "9c0ed608e33d4650842ba5a8f7388a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6644672d0cc245b5b24724c1289b3ddb" is_pre_election: true
I20260812 03:48:14.747465 20191 raft_consensus.cc:2454] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c0ed608e33d4650842ba5a8f7388a31 in term 0.
I20260812 03:48:14.747489 20778 raft_consensus.cc:2781] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:14.747529 20778 raft_consensus.cc:482] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:14.747560 20778 raft_consensus.cc:3037] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.748221 20778 raft_consensus.cc:504] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.748394 20778 leader_election.cc:288] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [CANDIDATE]: Term 1 election: Requested vote from peers 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569), 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933)
I20260812 03:48:14.748522 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7764783aaa104d2484146ede82c5e82d" candidate_uuid: "9c0ed608e33d4650842ba5a8f7388a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6644672d0cc245b5b24724c1289b3ddb"
I20260812 03:48:14.748598 20191 raft_consensus.cc:3037] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.748642 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7764783aaa104d2484146ede82c5e82d" candidate_uuid: "9c0ed608e33d4650842ba5a8f7388a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a6a2c621bc43ebaad7400db9a3072c"
I20260812 03:48:14.748744 20430 raft_consensus.cc:3037] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.749218 20191 raft_consensus.cc:2454] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c0ed608e33d4650842ba5a8f7388a31 in term 1.
I20260812 03:48:14.749357 20430 raft_consensus.cc:2454] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c0ed608e33d4650842ba5a8f7388a31 in term 1.
I20260812 03:48:14.749393 20538 leader_election.cc:302] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6644672d0cc245b5b24724c1289b3ddb, 9c0ed608e33d4650842ba5a8f7388a31; no voters:
I20260812 03:48:14.749462 20778 raft_consensus.cc:2781] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:14.749579 20778 raft_consensus.cc:686] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 LEADER]: Becoming Leader. State: Replica: 9c0ed608e33d4650842ba5a8f7388a31, State: Running, Role: LEADER
I20260812 03:48:14.749670 20778 consensus_queue.cc:237] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.749996 20829 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: TxnStatusTablet state changed. Reason: New leader 9c0ed608e33d4650842ba5a8f7388a31. Latest consensus state: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.750102 20829 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:48:14.750296 20832 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:48:14.750355 19927 catalog_manager.cc:5495] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } health_report { overall_health: HEALTHY } } }
I20260812 03:48:14.750425 20832 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:48:14.756397 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.753778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.7.132.131:33649)
0812 03:48:14.753812 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.753814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.753838 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:48:14.753919 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.753920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.753932 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.753982 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:14.754091 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.754551 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.755649 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.755836 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.755883 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.756073 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.756191 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.756261 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.756329 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.756343 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:48:14.756448 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.753715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33649 (local address 127.0.0.1:39218)
0812 03:48:14.753796 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.753806 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.753856 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.753988 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.753988 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.753989 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:48:14.754033 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.754035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.754644 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.754645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.755557 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.755561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.755907 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.755908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.755972 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.755989 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.756261 (+ 272us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.756270 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.756326 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.756375 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 03:48:14.757340 20430 raft_consensus.cc:1261] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Refusing update from remote peer 9c0ed608e33d4650842ba5a8f7388a31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:14.757447 20191 raft_consensus.cc:1261] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Refusing update from remote peer 9c0ed608e33d4650842ba5a8f7388a31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:14.757669 20829 consensus_queue.cc:1035] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:14.757798 20778 consensus_queue.cc:1035] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:14.758368 20777 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: TxnStatusTablet state changed. Reason: New leader 9c0ed608e33d4650842ba5a8f7388a31. Latest consensus state: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.758451 20777 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:48:14.758450 20794 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: TxnStatusTablet state changed. Reason: New leader 9c0ed608e33d4650842ba5a8f7388a31. Latest consensus state: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.758534 20794 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:48:14.759613 20778 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.759699 20778 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:48:14.759902 20794 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.759989 20794 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 6644672d0cc245b5b24724c1289b3ddb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:48:14.760700 20778 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.760788 20778 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 9c0ed608e33d4650842ba5a8f7388a31: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:48:14.760890 20777 tablet_replica.cc:439] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } } }
I20260812 03:48:14.760979 20777 tablet_replica.cc:442] T 7764783aaa104d2484146ede82c5e82d P 03a6a2c621bc43ebaad7400db9a3072c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:48:14.763693 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.761316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43330 (local address 127.7.132.130:40933)
0812 03:48:14.761353 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.761355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.761361 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:48:14.761395 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.761395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.761413 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.761469 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:14.761629 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.762182 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.762995 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.763184 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.763248 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.763381 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.763469 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.763542 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.763606 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.763624 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:48:14.768692 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.766083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33649 (local address 127.7.132.130:39095)
0812 03:48:14.766265 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.766274 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.766322 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.766433 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.766433 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.766434 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:48:14.766476 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.766477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.767313 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.767314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.767886 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.767887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.768344 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.768345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.768389 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.768400 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.768573 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.768574 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.768607 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.768632 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":72,"threads_started":1}
I20260812 03:48:14.768692 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.766157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:39095 (local address 127.7.132.131:33649)
0812 03:48:14.766214 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.766216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.766332 (+ 116us) server_negotiation.cc:407] Connection header received
0812 03:48:14.766358 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.766358 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.766367 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.766432 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:14.766533 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.767243 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.767966 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.768259 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.768329 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.768474 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.768517 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.768567 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.768610 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.768626 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:48:14.771143 20191 raft_consensus.cc:1261] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Refusing update from remote peer 03a6a2c621bc43ebaad7400db9a3072c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:14.771221 20641 raft_consensus.cc:1261] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Refusing update from remote peer 03a6a2c621bc43ebaad7400db9a3072c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:14.771422 20813 consensus_queue.cc:1035] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:14.771621 20777 consensus_queue.cc:1035] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:14.779059 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.776788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:53411 (local address 127.7.132.130:40933)
0812 03:48:14.776839 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.776840 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.776969 (+ 129us) server_negotiation.cc:407] Connection header received
0812 03:48:14.777033 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.777034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.777041 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.777089 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:14.777219 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.777633 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.778365 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.778557 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.778626 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.778807 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.778840 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.778898 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.778980 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.778998 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:48:14.779093 20849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.776747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:40933 (local address 127.7.132.131:53411)
0812 03:48:14.776896 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:14.776905 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:14.776982 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:14.777092 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:14.777093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:14.777094 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:48:14.777152 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.777153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.777707 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.777709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.778302 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:14.778306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.778642 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.778643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:14.778700 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.778735 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.778906 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.778908 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:48:14.778962 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:48:14.779001 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260812 03:48:14.785609 19927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:14.785692 19927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:14.786322 19927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: Sending DeleteTablet for 3 replicas of tablet c6644b9694ae4ca189b4fd09f4262b57
I20260812 03:48:14.786574 20393 tablet_service.cc:1514] Processing DeleteTablet for tablet c6644b9694ae4ca189b4fd09f4262b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43318
I20260812 03:48:14.786644 20183 tablet_service.cc:1514] Processing DeleteTablet for tablet c6644b9694ae4ca189b4fd09f4262b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:35988
I20260812 03:48:14.786631 20618 tablet_service.cc:1514] Processing DeleteTablet for tablet c6644b9694ae4ca189b4fd09f4262b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:39202
I20260812 03:48:14.786679 20806 tablet_replica.cc:330] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: stopping tablet replica
I20260812 03:48:14.786715 20805 tablet_replica.cc:330] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: stopping tablet replica
I20260812 03:48:14.786748 20806 raft_consensus.cc:2229] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:14.786746 20804 tablet_replica.cc:330] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: stopping tablet replica
I20260812 03:48:14.786782 20805 raft_consensus.cc:2229] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:14.786796 20804 raft_consensus.cc:2229] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:14.786815 20805 raft_consensus.cc:2258] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:14.786829 20804 raft_consensus.cc:2258] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:14.786826 20806 raft_consensus.cc:2258] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:14.787150 20806 ts_tablet_manager.cc:1905] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.788769 20806 ts_tablet_manager.cc:1918] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:48:14.788829 20806 log.cc:1196] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/c6644b9694ae4ca189b4fd09f4262b57
I20260812 03:48:14.789038 20806 ts_tablet_manager.cc:1939] T c6644b9694ae4ca189b4fd09f4262b57 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting consensus metadata
I20260812 03:48:14.789427 19909 catalog_manager.cc:4841] TS 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933): tablet c6644b9694ae4ca189b4fd09f4262b57 (table test-table [id=a21e2b1b55ac462893bc5ba666c7d826]) successfully deleted
I20260812 03:48:14.789690 20804 ts_tablet_manager.cc:1905] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.790904 20805 ts_tablet_manager.cc:1905] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.791312 20804 ts_tablet_manager.cc:1918] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:48:14.791355 20804 log.cc:1196] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/c6644b9694ae4ca189b4fd09f4262b57
I20260812 03:48:14.791543 20804 ts_tablet_manager.cc:1939] T c6644b9694ae4ca189b4fd09f4262b57 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting consensus metadata
I20260812 03:48:14.791908 19909 catalog_manager.cc:4841] TS 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649): tablet c6644b9694ae4ca189b4fd09f4262b57 (table test-table [id=a21e2b1b55ac462893bc5ba666c7d826]) successfully deleted
I20260812 03:48:14.792506 20805 ts_tablet_manager.cc:1918] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:48:14.792557 20805 log.cc:1196] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/c6644b9694ae4ca189b4fd09f4262b57
I20260812 03:48:14.792774 20805 ts_tablet_manager.cc:1939] T c6644b9694ae4ca189b4fd09f4262b57 P 6644672d0cc245b5b24724c1289b3ddb: Deleting consensus metadata
I20260812 03:48:14.793109 19907 catalog_manager.cc:4841] TS 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569): tablet c6644b9694ae4ca189b4fd09f4262b57 (table test-table [id=a21e2b1b55ac462893bc5ba666c7d826]) successfully deleted
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:48:14.817807 20044 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:14.811108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.7.132.190:46293)
0812 03:48:14.811156 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.811158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.811299 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:48:14.811352 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.811353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.811364 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.811427 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.812190 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.812991 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.814074 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.814394 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.814462 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.817726 (+ 3264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46696: BlockingRecv error: recv got EOF from 127.0.0.1:46696 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 03:48:14 dist-test-slave-39l8 krb5kdc[19859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM
I20260812 03:48:14.826841 20044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.818964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.7.132.190:46293)
0812 03:48:14.819001 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.819002 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.819011 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:14.819092 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.819093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.819102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.819172 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:14.819737 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.820401 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.821305 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.821577 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.821643 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.824164 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:14.824166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:14.824166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:14.824173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:14.825691 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.826083 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.826084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.826085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.826138 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:14.826332 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:14.826332 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:14.826333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:14.826446 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:14.826530 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.826658 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.826764 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:48:14.828560 19927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:48:14.828739 19927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:14.830099 20183 tablet_service.cc:1467] Processing CreateTablet for tablet 252434379b9947c284b550779908b105 (DEFAULT_TABLE table=test-table [id=327c5678e65349f2833362a55b4fd7cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.830117 20618 tablet_service.cc:1467] Processing CreateTablet for tablet 252434379b9947c284b550779908b105 (DEFAULT_TABLE table=test-table [id=327c5678e65349f2833362a55b4fd7cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.830236 20183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252434379b9947c284b550779908b105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.830243 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252434379b9947c284b550779908b105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.830804 20393 tablet_service.cc:1467] Processing CreateTablet for tablet 252434379b9947c284b550779908b105 (DEFAULT_TABLE table=test-table [id=327c5678e65349f2833362a55b4fd7cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.830922 20393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 252434379b9947c284b550779908b105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.831615 20770 tablet_bootstrap.cc:492] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: Bootstrap starting.
I20260812 03:48:14.832051 20770 tablet_bootstrap.cc:654] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.832230 20771 tablet_bootstrap.cc:492] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: Bootstrap starting.
I20260812 03:48:14.832631 20771 tablet_bootstrap.cc:654] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.832643 20770 tablet_bootstrap.cc:492] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: No bootstrap required, opened a new log
I20260812 03:48:14.832698 20770 ts_tablet_manager.cc:1397] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.832796 20770 raft_consensus.cc:348] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.832856 20770 raft_consensus.cc:374] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.832875 20770 raft_consensus.cc:729] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c0ed608e33d4650842ba5a8f7388a31, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.832917 20770 consensus_queue.cc:260] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.833002 20770 ts_tablet_manager.cc:1428] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.834596 20775 tablet_bootstrap.cc:492] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: Bootstrap starting.
I20260812 03:48:14.834933 20775 tablet_bootstrap.cc:654] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.835490 20775 tablet_bootstrap.cc:492] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: No bootstrap required, opened a new log
I20260812 03:48:14.835551 20775 ts_tablet_manager.cc:1397] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:14.835673 20775 raft_consensus.cc:348] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.835745 20775 raft_consensus.cc:374] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.835768 20775 raft_consensus.cc:729] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6644672d0cc245b5b24724c1289b3ddb, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.835806 20775 consensus_queue.cc:260] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.835851 20771 tablet_bootstrap.cc:492] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: No bootstrap required, opened a new log
I20260812 03:48:14.835893 20771 ts_tablet_manager.cc:1397] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:48:14.835893 20775 ts_tablet_manager.cc:1428] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:14.836023 20771 raft_consensus.cc:348] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.836081 20771 raft_consensus.cc:374] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.836100 20771 raft_consensus.cc:729] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a6a2c621bc43ebaad7400db9a3072c, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.836146 20771 consensus_queue.cc:260] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.836226 20771 ts_tablet_manager.cc:1428] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.851452 20794 raft_consensus.cc:482] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:14.851539 20794 raft_consensus.cc:504] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.851675 20794 leader_election.cc:288] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933), 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649)
I20260812 03:48:14.851897 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252434379b9947c284b550779908b105" candidate_uuid: "6644672d0cc245b5b24724c1289b3ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a6a2c621bc43ebaad7400db9a3072c" is_pre_election: true
I20260812 03:48:14.851938 20641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252434379b9947c284b550779908b105" candidate_uuid: "6644672d0cc245b5b24724c1289b3ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c0ed608e33d4650842ba5a8f7388a31" is_pre_election: true
I20260812 03:48:14.851992 20430 raft_consensus.cc:2454] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6644672d0cc245b5b24724c1289b3ddb in term 0.
I20260812 03:48:14.852018 20641 raft_consensus.cc:2454] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6644672d0cc245b5b24724c1289b3ddb in term 0.
I20260812 03:48:14.852169 20096 leader_election.cc:302] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6644672d0cc245b5b24724c1289b3ddb, 9c0ed608e33d4650842ba5a8f7388a31; no voters:
I20260812 03:48:14.852286 20794 raft_consensus.cc:2781] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:14.852324 20794 raft_consensus.cc:482] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:14.852350 20794 raft_consensus.cc:3037] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.853185 20794 raft_consensus.cc:504] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.853314 20794 leader_election.cc:288] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [CANDIDATE]: Term 1 election: Requested vote from peers 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933), 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649)
I20260812 03:48:14.853519 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252434379b9947c284b550779908b105" candidate_uuid: "6644672d0cc245b5b24724c1289b3ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a6a2c621bc43ebaad7400db9a3072c"
I20260812 03:48:14.853529 20641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "252434379b9947c284b550779908b105" candidate_uuid: "6644672d0cc245b5b24724c1289b3ddb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c0ed608e33d4650842ba5a8f7388a31"
I20260812 03:48:14.853591 20641 raft_consensus.cc:3037] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.853593 20430 raft_consensus.cc:3037] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:14.854261 20641 raft_consensus.cc:2454] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6644672d0cc245b5b24724c1289b3ddb in term 1.
I20260812 03:48:14.854383 20430 raft_consensus.cc:2454] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6644672d0cc245b5b24724c1289b3ddb in term 1.
I20260812 03:48:14.854408 20096 leader_election.cc:302] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6644672d0cc245b5b24724c1289b3ddb, 9c0ed608e33d4650842ba5a8f7388a31; no voters:
I20260812 03:48:14.854475 20794 raft_consensus.cc:2781] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:14.854506 20794 raft_consensus.cc:686] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 1 LEADER]: Becoming Leader. State: Replica: 6644672d0cc245b5b24724c1289b3ddb, State: Running, Role: LEADER
I20260812 03:48:14.854547 20794 consensus_queue.cc:237] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.855060 19930 catalog_manager.cc:5495] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb reported cstate change: term changed from 0 to 1, leader changed from <none> to 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129). New cstate: current_term: 1 leader_uuid: "6644672d0cc245b5b24724c1289b3ddb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } health_report { overall_health: UNKNOWN } } }
I20260812 03:48:14.866740 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:14.863826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36004 (local address 127.7.132.129:41569)
0812 03:48:14.863868 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:14.863870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:14.863879 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:14.863926 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:14.863927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:14.863937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:14.864004 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:14.864109 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.864849 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.865952 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:14.866213 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:14.866256 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:14.866464 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:14.866500 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:14.866562 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:14.866627 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:48:14.866655 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:48:14.867359 20641 raft_consensus.cc:1261] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Refusing update from remote peer 6644672d0cc245b5b24724c1289b3ddb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:14.867410 20430 raft_consensus.cc:1261] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Refusing update from remote peer 6644672d0cc245b5b24724c1289b3ddb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:14.867596 20786 consensus_queue.cc:1035] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:14.867684 20794 consensus_queue.cc:1035] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:14.871042 19930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:14.871143 19930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:14.872426 19930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: Sending DeleteTablet for 3 replicas of tablet 252434379b9947c284b550779908b105
I20260812 03:48:14.872706 20393 tablet_service.cc:1514] Processing DeleteTablet for tablet 252434379b9947c284b550779908b105 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43318
I20260812 03:48:14.872745 20618 tablet_service.cc:1514] Processing DeleteTablet for tablet 252434379b9947c284b550779908b105 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:39202
I20260812 03:48:14.872810 20806 tablet_replica.cc:330] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: stopping tablet replica
I20260812 03:48:14.872831 20804 tablet_replica.cc:330] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: stopping tablet replica
I20260812 03:48:14.872856 20806 raft_consensus.cc:2229] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:14.872874 20804 raft_consensus.cc:2229] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:14.872893 20806 raft_consensus.cc:2258] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:14.872915 20804 raft_consensus.cc:2258] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:14.873417 20183 tablet_service.cc:1514] Processing DeleteTablet for tablet 252434379b9947c284b550779908b105 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:14 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:35988
I20260812 03:48:14.873500 20805 tablet_replica.cc:330] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: stopping tablet replica
I20260812 03:48:14.873541 20805 raft_consensus.cc:2229] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:14.873600 20805 raft_consensus.cc:2258] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:14.873806 19927 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:48:14.873943 19927 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:14.874364 20805 ts_tablet_manager.cc:1905] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.876025 20805 ts_tablet_manager.cc:1918] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:14.876094 20805 log.cc:1196] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/252434379b9947c284b550779908b105
I20260812 03:48:14.876322 20805 ts_tablet_manager.cc:1939] T 252434379b9947c284b550779908b105 P 6644672d0cc245b5b24724c1289b3ddb: Deleting consensus metadata
I20260812 03:48:14.876811 20618 tablet_service.cc:1467] Processing CreateTablet for tablet 7b565f5c764f44a7aaf9c82732514c31 (DEFAULT_TABLE table=test-table [id=c0082ac428f9461a8ad465912762c82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.876830 20393 tablet_service.cc:1467] Processing CreateTablet for tablet 7b565f5c764f44a7aaf9c82732514c31 (DEFAULT_TABLE table=test-table [id=c0082ac428f9461a8ad465912762c82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.876925 20618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b565f5c764f44a7aaf9c82732514c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.876937 20393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b565f5c764f44a7aaf9c82732514c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.878172 20771 tablet_bootstrap.cc:492] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: Bootstrap starting.
I20260812 03:48:14.878273 20770 tablet_bootstrap.cc:492] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: Bootstrap starting.
I20260812 03:48:14.878599 20771 tablet_bootstrap.cc:654] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.878686 20770 tablet_bootstrap.cc:654] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.879199 20183 tablet_service.cc:1467] Processing CreateTablet for tablet 7b565f5c764f44a7aaf9c82732514c31 (DEFAULT_TABLE table=test-table [id=c0082ac428f9461a8ad465912762c82f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:14.879295 20183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b565f5c764f44a7aaf9c82732514c31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:14.879904 20770 tablet_bootstrap.cc:492] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: No bootstrap required, opened a new log
I20260812 03:48:14.879951 20770 ts_tablet_manager.cc:1397] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:48:14.880079 20770 raft_consensus.cc:348] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.880136 20770 raft_consensus.cc:374] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.880153 20770 raft_consensus.cc:729] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c0ed608e33d4650842ba5a8f7388a31, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.880192 20770 consensus_queue.cc:260] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.880270 20770 ts_tablet_manager.cc:1428] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.880451 20775 tablet_bootstrap.cc:492] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: Bootstrap starting.
I20260812 03:48:14.880847 20775 tablet_bootstrap.cc:654] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:14.881166 19907 catalog_manager.cc:4841] TS 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569): tablet 252434379b9947c284b550779908b105 (table test-table [id=327c5678e65349f2833362a55b4fd7cd]) successfully deleted
I20260812 03:48:14.881834 20806 ts_tablet_manager.cc:1905] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.881920 20804 ts_tablet_manager.cc:1905] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:14.883324 20804 ts_tablet_manager.cc:1918] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:14.883366 20806 ts_tablet_manager.cc:1918] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:14.883387 20804 log.cc:1196] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/252434379b9947c284b550779908b105
I20260812 03:48:14.883420 20806 log.cc:1196] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/252434379b9947c284b550779908b105
I20260812 03:48:14.883620 20804 ts_tablet_manager.cc:1939] T 252434379b9947c284b550779908b105 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting consensus metadata
I20260812 03:48:14.883703 20806 ts_tablet_manager.cc:1939] T 252434379b9947c284b550779908b105 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting consensus metadata
I20260812 03:48:14.883930 19909 catalog_manager.cc:4841] TS 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649): tablet 252434379b9947c284b550779908b105 (table test-table [id=327c5678e65349f2833362a55b4fd7cd]) successfully deleted
I20260812 03:48:14.884225 19909 catalog_manager.cc:4841] TS 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933): tablet 252434379b9947c284b550779908b105 (table test-table [id=327c5678e65349f2833362a55b4fd7cd]) successfully deleted
I20260812 03:48:14.884392 20771 tablet_bootstrap.cc:492] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: No bootstrap required, opened a new log
I20260812 03:48:14.884439 20771 ts_tablet_manager.cc:1397] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260812 03:48:14.884568 20771 raft_consensus.cc:348] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.884666 20771 raft_consensus.cc:374] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.884688 20771 raft_consensus.cc:729] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a6a2c621bc43ebaad7400db9a3072c, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.884742 20771 consensus_queue.cc:260] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.884840 20771 ts_tablet_manager.cc:1428] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:14.884949 20775 tablet_bootstrap.cc:492] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: No bootstrap required, opened a new log
I20260812 03:48:14.884995 20775 ts_tablet_manager.cc:1397] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 03:48:14.885087 20775 raft_consensus.cc:348] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.885149 20775 raft_consensus.cc:374] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:14.885169 20775 raft_consensus.cc:729] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6644672d0cc245b5b24724c1289b3ddb, State: Initialized, Role: FOLLOWER
I20260812 03:48:14.885210 20775 consensus_queue.cc:260] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.885285 20775 ts_tablet_manager.cc:1428] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:14.998780 20829 raft_consensus.cc:482] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:14.998908 20829 raft_consensus.cc:504] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:14.999086 20829 leader_election.cc:288] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569), 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933)
I20260812 03:48:14.999321 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b565f5c764f44a7aaf9c82732514c31" candidate_uuid: "9c0ed608e33d4650842ba5a8f7388a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a6a2c621bc43ebaad7400db9a3072c" is_pre_election: true
I20260812 03:48:14.999321 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b565f5c764f44a7aaf9c82732514c31" candidate_uuid: "9c0ed608e33d4650842ba5a8f7388a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6644672d0cc245b5b24724c1289b3ddb" is_pre_election: true
I20260812 03:48:14.999424 20430 raft_consensus.cc:2454] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c0ed608e33d4650842ba5a8f7388a31 in term 0.
I20260812 03:48:14.999423 20191 raft_consensus.cc:2454] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c0ed608e33d4650842ba5a8f7388a31 in term 0.
I20260812 03:48:14.999565 20538 leader_election.cc:302] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6644672d0cc245b5b24724c1289b3ddb, 9c0ed608e33d4650842ba5a8f7388a31; no voters:
I20260812 03:48:14.999676 20829 raft_consensus.cc:2781] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:14.999715 20829 raft_consensus.cc:482] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:14.999739 20829 raft_consensus.cc:3037] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:15.000417 20829 raft_consensus.cc:504] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:15.000555 20829 leader_election.cc:288] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [CANDIDATE]: Term 1 election: Requested vote from peers 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569), 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933)
I20260812 03:48:15.000716 20191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b565f5c764f44a7aaf9c82732514c31" candidate_uuid: "9c0ed608e33d4650842ba5a8f7388a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6644672d0cc245b5b24724c1289b3ddb"
I20260812 03:48:15.000736 20430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b565f5c764f44a7aaf9c82732514c31" candidate_uuid: "9c0ed608e33d4650842ba5a8f7388a31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a6a2c621bc43ebaad7400db9a3072c"
I20260812 03:48:15.000790 20191 raft_consensus.cc:3037] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:15.000801 20430 raft_consensus.cc:3037] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:15.001555 20430 raft_consensus.cc:2454] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c0ed608e33d4650842ba5a8f7388a31 in term 1.
I20260812 03:48:15.001555 20191 raft_consensus.cc:2454] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c0ed608e33d4650842ba5a8f7388a31 in term 1.
I20260812 03:48:15.001727 20540 leader_election.cc:302] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a6a2c621bc43ebaad7400db9a3072c, 9c0ed608e33d4650842ba5a8f7388a31; no voters:
I20260812 03:48:15.001816 20829 raft_consensus.cc:2781] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:15.001873 20829 raft_consensus.cc:686] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 LEADER]: Becoming Leader. State: Replica: 9c0ed608e33d4650842ba5a8f7388a31, State: Running, Role: LEADER
I20260812 03:48:15.001937 20829 consensus_queue.cc:237] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } }
I20260812 03:48:15.002517 19927 catalog_manager.cc:5495] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "9c0ed608e33d4650842ba5a8f7388a31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c0ed608e33d4650842ba5a8f7388a31" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33649 } health_report { overall_health: HEALTHY } } }
I20260812 03:48:15.028395 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:15.026462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39222 (local address 127.7.132.131:33649)
0812 03:48:15.026500 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:15.026502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:15.026510 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:48:15.026560 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:15.026560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:15.026569 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:15.026615 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:15.026768 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:15.027230 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:15.027893 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:15.028060 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:15.028108 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:15.028218 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:15.028244 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:15.028279 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:15.028326 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 03:48:15.028341 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:48:15.032547 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:15.030323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49679 (local address 127.7.132.131:33649)
0812 03:48:15.030356 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:15.030358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:15.030366 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:48:15.030428 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:15.030428 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:15.030437 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:15.030486 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:15.030596 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:15.031167 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:15.031857 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:15.032091 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:15.032187 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:15.032326 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:15.032354 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:15.032400 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:15.032455 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:48:15.032473 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":16}
I20260812 03:48:15.032554 20849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:15.030266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33649 (local address 127.7.132.131:49679)
0812 03:48:15.030321 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:15.030331 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:15.030373 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:15.030489 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:15.030490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:15.030490 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:48:15.030526 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:15.030527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:15.031252 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:15.031253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:15.031758 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:15.031760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:15.032189 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:15.032190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:15.032243 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:15.032248 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:48:15.032399 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:48:15.032400 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 03:48:15.032452 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:48:15.032491 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:48:15.034281 20430 raft_consensus.cc:1261] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Refusing update from remote peer 9c0ed608e33d4650842ba5a8f7388a31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:15.034294 20191 raft_consensus.cc:1261] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Refusing update from remote peer 9c0ed608e33d4650842ba5a8f7388a31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:15.034475 20829 consensus_queue.cc:1035] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a6a2c621bc43ebaad7400db9a3072c" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:15.034554 20840 consensus_queue.cc:1035] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6644672d0cc245b5b24724c1289b3ddb" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:15.044683 19929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:15.044782 19929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:15.045403 19929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b109da4fdd0344038ce94f8cf1753d6f: Sending DeleteTablet for 3 replicas of tablet 7b565f5c764f44a7aaf9c82732514c31
I20260812 03:48:15.045706 20393 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b565f5c764f44a7aaf9c82732514c31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:15 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43318
I20260812 03:48:15.045703 20183 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b565f5c764f44a7aaf9c82732514c31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:15 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:35988
I20260812 03:48:15.045804 20805 tablet_replica.cc:330] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: stopping tablet replica
I20260812 03:48:15.045858 20805 raft_consensus.cc:2229] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:15.045864 20806 tablet_replica.cc:330] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: stopping tablet replica
I20260812 03:48:15.045894 20805 raft_consensus.cc:2258] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:15.045912 20806 raft_consensus.cc:2229] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:15.045954 20806 raft_consensus.cc:2258] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:15.046326 20805 ts_tablet_manager.cc:1905] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:15.046476 20806 ts_tablet_manager.cc:1905] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:15.047814 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 20058
I20260812 03:48:15.047878 20805 ts_tablet_manager.cc:1918] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:48:15.047928 20805 log.cc:1196] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/7b565f5c764f44a7aaf9c82732514c31
I20260812 03:48:15.048157 20805 ts_tablet_manager.cc:1939] T 7b565f5c764f44a7aaf9c82732514c31 P 6644672d0cc245b5b24724c1289b3ddb: Deleting consensus metadata
I20260812 03:48:15.054304 20806 ts_tablet_manager.cc:1918] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:48:15.054373 20806 log.cc:1196] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/7b565f5c764f44a7aaf9c82732514c31
I20260812 03:48:15.054587 20806 ts_tablet_manager.cc:1939] T 7b565f5c764f44a7aaf9c82732514c31 P 03a6a2c621bc43ebaad7400db9a3072c: Deleting consensus metadata
I20260812 03:48:15.054910 20618 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b565f5c764f44a7aaf9c82732514c31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:15 UTC) from {username='oryx', principal='oryx/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:39202
I20260812 03:48:15.054989 20804 tablet_replica.cc:330] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: stopping tablet replica
I20260812 03:48:15.054984 19909 catalog_manager.cc:4841] TS 03a6a2c621bc43ebaad7400db9a3072c (127.7.132.130:40933): tablet 7b565f5c764f44a7aaf9c82732514c31 (table test-table [id=c0082ac428f9461a8ad465912762c82f]) successfully deleted
I20260812 03:48:15.055040 20804 raft_consensus.cc:2229] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:15.055121 20804 raft_consensus.cc:2258] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:15.055475 20804 ts_tablet_manager.cc:1905] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:15.056936 20804 ts_tablet_manager.cc:1918] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:48:15.057005 20804 log.cc:1196] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/7b565f5c764f44a7aaf9c82732514c31
I20260812 03:48:15.057216 20804 ts_tablet_manager.cc:1939] T 7b565f5c764f44a7aaf9c82732514c31 P 9c0ed608e33d4650842ba5a8f7388a31: Deleting consensus metadata
I20260812 03:48:15.058933 19909 catalog_manager.cc:4841] TS 9c0ed608e33d4650842ba5a8f7388a31 (127.7.132.131:33649): tablet 7b565f5c764f44a7aaf9c82732514c31 (table test-table [id=c0082ac428f9461a8ad465912762c82f]) successfully deleted
W20260812 03:48:15.062377 19907 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:41569 (error 108)
I20260812 03:48:15.062981 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:15.062790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:41569 (local address 127.0.0.1:36008)
0812 03:48:15.062889 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:15.062933 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":57,"threads_started":1}
I20260812 03:48:15.063019 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 20295
W20260812 03:48:15.073406 19907 catalog_manager.cc:4584] TS 6644672d0cc245b5b24724c1289b3ddb (127.7.132.129:41569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b565f5c764f44a7aaf9c82732514c31: Network error: Client connection negotiation failed: client connection to 127.7.132.129:41569: connect: Connection refused (error 111)
I20260812 03:48:15.081938 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 20513
I20260812 03:48:15.094274 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 19883
2026-08-12T03:48:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2832 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:48:15 dist-test-slave-39l8 krb5kdc[20906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:48:15 dist-test-slave-39l8 krb5kdc[20906](info): set up 2 sockets
Aug 12 03:48:15 dist-test-slave-39l8 krb5kdc[20906](info): commencing operation
krb5kdc: starting...
W20260812 03:48:16.293067 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:48:16 dist-test-slave-39l8 krb5kdc[20906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:48:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:48:16.403739 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:35867
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:35867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:16.484154 20927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:16.484359 20927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:16.484390 20927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:16.485497 20927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:16.485560 20927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:16.485595 20927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:16.485625 20927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:16.485651 20927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:16.487188 20927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:35867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:35867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:16.487455 20927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:16.487673 20927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:16.492286 20934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:16.492332 20933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:16.492432 20927 server_base.cc:1034] running on GCE node
W20260812 03:48:16.492535 20940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:16.492802 20927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:16.493101 20927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:16.494297 20927 hybrid_clock.cc:648] HybridClock initialized: now 1786506496494283 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T03:48:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1401 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:48:16 dist-test-slave-39l8 krb5kdc[20949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:48:16 dist-test-slave-39l8 krb5kdc[20949](info): set up 2 sockets
Aug 12 03:48:16 dist-test-slave-39l8 krb5kdc[20949](info): commencing operation
krb5kdc: starting...
W20260812 03:48:17.698803 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:48:17 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:48:17.810745 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34061
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:34061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:17.888518 20968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:17.888698 20968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:17.888718 20968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:17.889760 20968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:17.889799 20968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:17.889811 20968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:17.889823 20968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:17.889902 20968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:17.891351 20968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:34061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:17.891577 20968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:17.891816 20968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:17.896488 20974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:17.896545 20968 server_base.cc:1034] running on GCE node
W20260812 03:48:17.896513 20978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:17.896522 20975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:17.896960 20968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:17.897267 20968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:17.898440 20968 hybrid_clock.cc:648] HybridClock initialized: now 1786506497898429 us; error 35 us; skew 500 ppm
Aug 12 03:48:17 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506497, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:17.900846 20968 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:48:17.901281 20968 webserver.cc:460] Webserver started at http://127.7.132.190:38829/ using document root <none> and password file <none>
I20260812 03:48:17.901494 20968 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:17.901556 20968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:17.901682 20968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:17.902815 20968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "b787d28df1bf4aa58ce8984b3559ee2b"
format_stamp: "Formatted at 2026-08-12 03:48:17 on dist-test-slave-39l8"
server_key: "67878c4cb805a9d71c2194dc43dbc542"
server_key_iv: "6d10be27dee3be47159f409a5d6f935a"
server_key_version: "encryptionkey@0"
I20260812 03:48:17.903196 20968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "b787d28df1bf4aa58ce8984b3559ee2b"
format_stamp: "Formatted at 2026-08-12 03:48:17 on dist-test-slave-39l8"
server_key: "67878c4cb805a9d71c2194dc43dbc542"
server_key_iv: "6d10be27dee3be47159f409a5d6f935a"
server_key_version: "encryptionkey@0"
I20260812 03:48:17.904502 20968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:48:17.905246 20990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:17.905476 20968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:17.905606 20968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "b787d28df1bf4aa58ce8984b3559ee2b"
format_stamp: "Formatted at 2026-08-12 03:48:17 on dist-test-slave-39l8"
server_key: "67878c4cb805a9d71c2194dc43dbc542"
server_key_iv: "6d10be27dee3be47159f409a5d6f935a"
server_key_version: "encryptionkey@0"
I20260812 03:48:17.905689 20968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:17.915220 20968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:17.916844 20968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:17.917002 20968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:17.921037 20968 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:34061
I20260812 03:48:17.921075 21066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:34061 every 8 connection(s)
I20260812 03:48:17.921460 20968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:17.922386 21068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:17.924649 21068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Bootstrap starting.
I20260812 03:48:17.925135 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 20968
I20260812 03:48:17.925258 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:17.925331 21068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:17.925472 7698 external_mini_cluster.cc:1428] Setting key 4dada666922f83fd360bbef669f1ef68
I20260812 03:48:17.925637 21068 log.cc:824] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:17.926337 21068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: No bootstrap required, opened a new log
I20260812 03:48:17.928218 21068 raft_consensus.cc:348] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } }
I20260812 03:48:17.928366 21068 raft_consensus.cc:374] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:17.928411 21068 raft_consensus.cc:729] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b787d28df1bf4aa58ce8984b3559ee2b, State: Initialized, Role: FOLLOWER
I20260812 03:48:17.928532 21068 consensus_queue.cc:260] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } }
I20260812 03:48:17.928592 21068 raft_consensus.cc:388] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:17.928611 21068 raft_consensus.cc:482] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:17.928666 21068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:17.929141 21068 raft_consensus.cc:504] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } }
I20260812 03:48:17.929265 21068 leader_election.cc:302] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b787d28df1bf4aa58ce8984b3559ee2b; no voters:
I20260812 03:48:17.929450 21068 leader_election.cc:288] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:17.929489 21075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:17.929603 21075 raft_consensus.cc:686] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 1 LEADER]: Becoming Leader. State: Replica: b787d28df1bf4aa58ce8984b3559ee2b, State: Running, Role: LEADER
I20260812 03:48:17.929710 21068 sys_catalog.cc:564] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:17.929737 21075 consensus_queue.cc:237] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } }
I20260812 03:48:17.930032 21075 sys_catalog.cc:455] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b787d28df1bf4aa58ce8984b3559ee2b. Latest consensus state: current_term: 1 leader_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } } }
I20260812 03:48:17.930043 21080 sys_catalog.cc:455] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } } }
I20260812 03:48:17.930169 21075 sys_catalog.cc:458] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:17.930178 21080 sys_catalog.cc:458] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:17.930975 21097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:17.931526 21097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:48:17 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:17.933965 21097 catalog_manager.cc:1347] Generated new cluster ID: a6e15965fefd4409a39ced9119465db2
I20260812 03:48:17.934017 21097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:17.936084 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:17.925956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.7.132.190:34061)
0812 03:48:17.926110 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:17.926119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:17.926151 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:48:17.926612 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:17.926619 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:17.926671 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:17.926828 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:17.927434 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:17.928293 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:17.929121 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:17.929699 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:17.930065 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:17.932798 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:17.932802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:17.932803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:17.932815 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:17.934524 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:17.934948 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:17.934956 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:17.934956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:17.935044 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:17.935247 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:17.935247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:17.935248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:17.935464 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:17.935637 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:17.935737 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:48:17.935869 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260812 03:48:17.982722 21097 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:17.983429 21097 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:17.990087 21097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Generated new TSK 0
I20260812 03:48:17.990280 21097 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:48:18.023428 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34061
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:18.121495 21114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:18.121671 21114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:18.121693 21114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:18.122781 21114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:18.122812 21114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:18.122838 21114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:18.124174 21114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:18.124346 21114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:18.124513 21114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:48:18.129230 21114 server_base.cc:1034] running on GCE node
W20260812 03:48:18.129209 21122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.129209 21125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.129199 21127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:18.130275 21114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:18.130537 21114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:18.131685 21114 hybrid_clock.cc:648] HybridClock initialized: now 1786506498131674 us; error 28 us; skew 500 ppm
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:18.134037 21114 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:48:18.134519 21114 webserver.cc:460] Webserver started at http://127.7.132.129:38237/ using document root <none> and password file <none>
I20260812 03:48:18.134786 21114 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:18.134853 21114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:18.134985 21114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:18.135964 21114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "16191460648f4378ac1a42f1a43e95d4"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "060b16469147bd15224df47680a901ae"
server_key_iv: "2ea0c5a2330811a2f2d89a021adef2af"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.136356 21114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "16191460648f4378ac1a42f1a43e95d4"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "060b16469147bd15224df47680a901ae"
server_key_iv: "2ea0c5a2330811a2f2d89a021adef2af"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.137621 21114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:18.138329 21143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.138486 21114 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:18.138589 21114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "16191460648f4378ac1a42f1a43e95d4"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "060b16469147bd15224df47680a901ae"
server_key_iv: "2ea0c5a2330811a2f2d89a021adef2af"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.138675 21114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:18.172566 21114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:18.174330 21114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:18.174490 21114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:18.174741 21114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:18.175041 21114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:18.175099 21114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.175132 21114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:18.175172 21114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.180955 21114 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45279
I20260812 03:48:18.181006 21323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45279 every 8 connection(s)
I20260812 03:48:18.181351 21114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:18.189416 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 21114
I20260812 03:48:18.189541 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:18.189760 7698 external_mini_cluster.cc:1428] Setting key 2c213c6cbb6d973f0867de5caa832b84
I20260812 03:48:18.189782 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.181992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:37929 (local address 127.7.132.190:34061)
0812 03:48:18.182061 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.182063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.182346 (+ 283us) server_negotiation.cc:407] Connection header received
0812 03:48:18.182851 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.182852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.182863 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.182917 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.183793 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.184262 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.185176 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.185383 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.185431 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.187392 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:18.187393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:18.187393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:18.187399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:18.188544 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:18.188890 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:18.188890 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:18.188890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:18.188939 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:18.189139 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:18.189139 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:18.189140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:18.189231 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:18.189281 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:18.189619 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 03:48:18.189713 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:48:18.190110 21330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.181988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:37929)
0812 03:48:18.182217 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.182253 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.182695 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.182994 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.182997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.183537 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.183705 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.183709 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.184336 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.184336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.185098 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.185100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.185453 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.185453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.185541 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.185933 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.185940 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.187316 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:18.188611 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:18.188617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:18.188619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:18.188827 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:18.188996 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:18.188996 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:18.188996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:18.189087 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:18.189297 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:18.189304 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:18.189461 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 03:48:18.189584 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 03:48:18.189755 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":102,"threads_started":1}
I20260812 03:48:18.190577 21327 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34061
I20260812 03:48:18.190682 21327 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:18.190893 21327 heartbeater.cc:511] Master 127.7.132.190:34061 requested a full tablet report, sending...
I20260812 03:48:18.191457 21010 ts_manager.cc:194] Registered new tserver with Master: 16191460648f4378ac1a42f1a43e95d4 (127.7.132.129:45279)
I20260812 03:48:18.192360 21010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:37929
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:48:18.214512 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34061
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:18.296222 21335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:18.296409 21335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:18.296432 21335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:18.297461 21335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:18.297495 21335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:18.297519 21335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:18.298849 21335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:18.299041 21335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:18.299207 21335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:18.304133 21344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.304103 21347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.304118 21343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:18.304312 21335 server_base.cc:1034] running on GCE node
I20260812 03:48:18.304514 21335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:18.304720 21335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:18.305860 21335 hybrid_clock.cc:648] HybridClock initialized: now 1786506498305841 us; error 40 us; skew 500 ppm
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:18.308364 21335 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:48:18.308864 21335 webserver.cc:460] Webserver started at http://127.7.132.130:33607/ using document root <none> and password file <none>
I20260812 03:48:18.309087 21335 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:18.309154 21335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:18.309306 21335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:18.310331 21335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "ed42b3d4a3cb41bb991c53c85daf3edb"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "b35d30421e965c2b19d53287aa197110"
server_key_iv: "e586c6f2f7c2a106f79ea5ffe5275128"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.310691 21335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "ed42b3d4a3cb41bb991c53c85daf3edb"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "b35d30421e965c2b19d53287aa197110"
server_key_iv: "e586c6f2f7c2a106f79ea5ffe5275128"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.311993 21335 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:48:18.312752 21358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.312927 21335 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:18.313028 21335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "ed42b3d4a3cb41bb991c53c85daf3edb"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "b35d30421e965c2b19d53287aa197110"
server_key_iv: "e586c6f2f7c2a106f79ea5ffe5275128"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.313113 21335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:18.324334 21335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:18.326056 21335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:18.326216 21335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:18.326457 21335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:18.326826 21335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:18.326881 21335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.326917 21335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:18.326943 21335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.332916 21335 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:35457
I20260812 03:48:18.333007 21533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:35457 every 8 connection(s)
I20260812 03:48:18.333304 21335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:18.338795 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 21335
I20260812 03:48:18.338929 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:18.339149 7698 external_mini_cluster.cc:1428] Setting key 99771a6834bc760133ff18ad80335b3a
I20260812 03:48:18.342478 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.334008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:53499 (local address 127.7.132.190:34061)
0812 03:48:18.334066 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.334068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.334259 (+ 191us) server_negotiation.cc:407] Connection header received
0812 03:48:18.334733 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.334734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.334747 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.334813 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.335692 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.336164 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.337074 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.337265 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.337319 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.339573 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:18.339575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:18.339575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:18.339583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:18.341052 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:18.341493 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:18.341494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:18.341494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:18.341560 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:18.341795 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:18.341796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:18.341796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:18.341914 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:18.341990 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:18.342304 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 03:48:18.342390 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:48:18.342656 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.333961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:53499)
0812 03:48:18.334121 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.334169 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.334625 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.334875 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.334879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.335414 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.335603 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.335608 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.336242 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.336243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.336993 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.336995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.337339 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.337339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.337433 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.337947 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.337955 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.339462 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:18.341182 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:18.341188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:18.341190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:18.341404 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:18.341627 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:18.341628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:18.341628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:18.341727 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:18.341981 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:18.341988 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:18.342111 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 03:48:18.342276 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 03:48:18.342431 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260812 03:48:18.343252 21534 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34061
I20260812 03:48:18.343329 21534 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:18.343516 21534 heartbeater.cc:511] Master 127.7.132.190:34061 requested a full tablet report, sending...
I20260812 03:48:18.343901 21010 ts_manager.cc:194] Registered new tserver with Master: ed42b3d4a3cb41bb991c53c85daf3edb (127.7.132.130:35457)
I20260812 03:48:18.344224 21010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:53499
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:48:18.363390 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34061
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:18.443195 21539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:18.443387 21539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:18.443409 21539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:18.444433 21539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:18.444471 21539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:18.444520 21539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:18.445931 21539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:18.446161 21539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:18.446378 21539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:18.451220 21547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.451236 21550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:18.451303 21539 server_base.cc:1034] running on GCE node
W20260812 03:48:18.451236 21548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:18.451655 21539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:18.451905 21539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:18.453068 21539 hybrid_clock.cc:648] HybridClock initialized: now 1786506498453055 us; error 40 us; skew 500 ppm
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:18.455530 21539 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:48:18.455943 21539 webserver.cc:460] Webserver started at http://127.7.132.131:42363/ using document root <none> and password file <none>
I20260812 03:48:18.456123 21539 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:18.456167 21539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:18.456248 21539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:18.457144 21539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "be1c0d83fec44c468a4b8661f6bc8ed2"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "6017ef437f0356d011c1b36da6032d18"
server_key_iv: "ad59e161ad2fb13d7223cce6e0bfee0f"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.457442 21539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "be1c0d83fec44c468a4b8661f6bc8ed2"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "6017ef437f0356d011c1b36da6032d18"
server_key_iv: "ad59e161ad2fb13d7223cce6e0bfee0f"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.458693 21539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:48:18.459452 21558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.459602 21539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:18.459682 21539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "be1c0d83fec44c468a4b8661f6bc8ed2"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "6017ef437f0356d011c1b36da6032d18"
server_key_iv: "ad59e161ad2fb13d7223cce6e0bfee0f"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.459741 21539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:18.469868 21539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:18.471572 21539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:18.471676 21539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:18.471837 21539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:18.472143 21539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:18.472184 21539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.472204 21539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:18.472250 21539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.477659 21539 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:39755
I20260812 03:48:18.477703 21746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:39755 every 8 connection(s)
I20260812 03:48:18.477993 21539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:18.486516 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.478652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41641 (local address 127.7.132.190:34061)
0812 03:48:18.478720 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.478722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.478897 (+ 175us) server_negotiation.cc:407] Connection header received
0812 03:48:18.479374 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.479375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.479389 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.479451 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.480305 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.480800 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.481785 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.481989 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.482061 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.484102 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:18.484103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:18.484104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:18.484111 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:18.485221 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:18.485608 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:18.485609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:18.485610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:18.485668 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:18.485907 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:18.485908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:18.485909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:18.486022 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:18.486097 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:18.486352 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 03:48:18.486432 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:48:18.486672 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.478598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:41641)
0812 03:48:18.478787 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.478820 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.479270 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.479519 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.479523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.480027 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.480190 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.480194 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.480893 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.480894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.481659 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.481661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.482078 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.482080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.482200 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.482626 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.482634 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.484004 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:18.485311 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:18.485317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:18.485320 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:18.485523 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:18.485740 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:18.485741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:18.485741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:18.485838 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:18.486088 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:18.486094 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:18.486221 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 03:48:18.486321 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:48:18.486465 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":67,"threads_started":1}
I20260812 03:48:18.487165 21747 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34061
I20260812 03:48:18.487253 21747 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:18.487380 21747 heartbeater.cc:511] Master 127.7.132.190:34061 requested a full tablet report, sending...
I20260812 03:48:18.487710 21010 ts_manager.cc:194] Registered new tserver with Master: be1c0d83fec44c468a4b8661f6bc8ed2 (127.7.132.131:39755)
I20260812 03:48:18.487856 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 21539
I20260812 03:48:18.487922 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:18.488087 7698 external_mini_cluster.cc:1428] Setting key 4a3dc56955297cfa3beb99478c290732
I20260812 03:48:18.488134 21010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:41641
I20260812 03:48:18.488999 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
Entry for principal oryx/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab.
WARNING: no policy specified for oryx/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
Entry for principal oryx/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab.
WARNING: no policy specified for oryx/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
Entry for principal oryx/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab.
I20260812 03:48:18.554316 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 21114
I20260812 03:48:18.566905 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 21335
I20260812 03:48:18.582412 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 21539
I20260812 03:48:18.592221 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 20968
I20260812 03:48:18.602900 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34061
--webserver_interface=127.7.132.190
--webserver_port=38829
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:34061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:18.681665 21758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:18.681840 21758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:18.681861 21758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:18.682921 21758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:18.682953 21758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:18.682967 21758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:18.682979 21758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:18.682991 21758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:18.684304 21758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:34061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=38829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:18.684469 21758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:18.684618 21758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:18.689030 21768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.689043 21763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.689047 21764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:18.689370 21758 server_base.cc:1034] running on GCE node
I20260812 03:48:18.689580 21758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:18.689800 21758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:18.691087 21758 hybrid_clock.cc:648] HybridClock initialized: now 1786506498691076 us; error 38 us; skew 500 ppm
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:18.693507 21758 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:48:18.693918 21758 webserver.cc:460] Webserver started at http://127.7.132.190:38829/ using document root <none> and password file <none>
I20260812 03:48:18.694164 21758 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:18.694253 21758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:18.695565 21758 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:18.696161 21781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.696349 21758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:18.696448 21758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "b787d28df1bf4aa58ce8984b3559ee2b"
format_stamp: "Formatted at 2026-08-12 03:48:17 on dist-test-slave-39l8"
server_key: "67878c4cb805a9d71c2194dc43dbc542"
server_key_iv: "6d10be27dee3be47159f409a5d6f935a"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.696735 21758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:18.708464 21758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:18.710005 21758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:18.710155 21758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:18.714221 21758 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:34061
I20260812 03:48:18.714267 21856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:34061 every 8 connection(s)
I20260812 03:48:18.714619 21758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:18.715914 21857 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:48:18.716519 21857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Bootstrap starting.
I20260812 03:48:18.717943 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 21758
I20260812 03:48:18.718299 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:45279
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:34061
--webserver_port=38237
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.132.129
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:48:18.725966 21857 log.cc:824] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:18.727492 21857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:48:18.727777 21857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Bootstrap complete.
I20260812 03:48:18.729885 21857 raft_consensus.cc:348] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } }
I20260812 03:48:18.730119 21857 raft_consensus.cc:729] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b787d28df1bf4aa58ce8984b3559ee2b, State: Initialized, Role: FOLLOWER
I20260812 03:48:18.730289 21857 consensus_queue.cc:260] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } }
I20260812 03:48:18.730362 21857 raft_consensus.cc:388] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:18.730412 21857 raft_consensus.cc:482] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:18.730458 21857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:48:18.731494 21857 raft_consensus.cc:504] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } }
I20260812 03:48:18.731580 21857 leader_election.cc:302] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b787d28df1bf4aa58ce8984b3559ee2b; no voters:
I20260812 03:48:18.731761 21857 leader_election.cc:288] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:48:18.731793 21864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:48:18.731913 21864 raft_consensus.cc:686] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [term 2 LEADER]: Becoming Leader. State: Replica: b787d28df1bf4aa58ce8984b3559ee2b, State: Running, Role: LEADER
I20260812 03:48:18.732038 21857 sys_catalog.cc:564] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:18.732098 21864 consensus_queue.cc:237] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } }
I20260812 03:48:18.732563 21866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b787d28df1bf4aa58ce8984b3559ee2b. Latest consensus state: current_term: 2 leader_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } } }
I20260812 03:48:18.732656 21866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:18.732574 21867 sys_catalog.cc:455] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b787d28df1bf4aa58ce8984b3559ee2b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34061 } } }
I20260812 03:48:18.732702 21867 sys_catalog.cc:458] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:18.732847 21874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:18.733429 21874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:18.733628 21874 catalog_manager.cc:1259] Loaded cluster ID: a6e15965fefd4409a39ced9119465db2
I20260812 03:48:18.733816 21874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:18.735157 21874 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:18.735328 21874 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b787d28df1bf4aa58ce8984b3559ee2b: Loaded TSK: 0
I20260812 03:48:18.735529 21874 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:48:18.800876 21861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:18.801054 21861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:18.801076 21861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:18.802148 21861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:18.802184 21861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:18.802212 21861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:18.803689 21861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.129.keytab
--principal=oryx/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:45279
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=38237
--tserver_master_addrs=127.7.132.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:18.803903 21861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:18.804136 21861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:18.809012 21899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.809038 21896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.809056 21897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:18.809433 21861 server_base.cc:1034] running on GCE node
I20260812 03:48:18.809651 21861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:18.809896 21861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:18.811053 21861 hybrid_clock.cc:648] HybridClock initialized: now 1786506498811040 us; error 32 us; skew 500 ppm
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:18.813771 21861 init.cc:377] Logged in from keytab as oryx/127.7.132.129@KRBTEST.COM (short username oryx)
I20260812 03:48:18.814275 21861 webserver.cc:460] Webserver started at http://127.7.132.129:38237/ using document root <none> and password file <none>
I20260812 03:48:18.814505 21861 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:18.814580 21861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:18.816383 21861 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:18.817037 21910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.817211 21861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:18.817305 21861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "16191460648f4378ac1a42f1a43e95d4"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "060b16469147bd15224df47680a901ae"
server_key_iv: "2ea0c5a2330811a2f2d89a021adef2af"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.817703 21861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:18.832633 21861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:18.835206 21861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:18.835354 21861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:18.835606 21861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:18.835945 21861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:18.835995 21861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.836025 21861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:18.836047 21861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.843035 21861 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45279
I20260812 03:48:18.843063 22074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45279 every 8 connection(s)
I20260812 03:48:18.843482 21861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:48:18.844208 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 21861
I20260812 03:48:18.844542 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:35457
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:34061
--webserver_port=33607
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.132.130
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:18.855263 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.844223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:46353)
0812 03:48:18.844432 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.844475 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.845054 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.846833 (+ 1779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.846837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.847593 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.847840 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.847845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.849459 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.849460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.850835 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.850839 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.851601 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.851602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.851756 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.852357 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.852368 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.855097 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
W20260812 03:48:18.855522 22075 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:34061 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.132.190:34061: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
W20260812 03:48:18.855614 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:18.846155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:46353 (local address 127.7.132.190:34061)
0812 03:48:18.846364 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.846373 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.846411 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:48:18.846528 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.846534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.846586 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.846776 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.847972 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.849343 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.850937 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.851500 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.851607 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.855255 (+ 3648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:46353: BlockingRecv error: recv got EOF from 127.7.132.129:46353 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:48:18.861048 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:18.855889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:49941 (local address 127.7.132.190:34061)
0812 03:48:18.856160 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.856162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.856170 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:48:18.856263 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.856264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.856273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.856299 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.856611 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.857315 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.858365 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.858627 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.858660 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.860997 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:49941: BlockingRecv error: recv got EOF from 127.7.132.129:49941 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:48:18.861212 22084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.855716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:49941)
0812 03:48:18.855867 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.855877 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.856193 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.856348 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.856349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.856498 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.856541 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.856542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.857407 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.857408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.858267 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.858268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.858811 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.858813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.858870 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.859264 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.859268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.861146 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:18.866138 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.861398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:39097)
0812 03:48:18.861428 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.861435 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.861478 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.861605 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.861606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.861753 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.861781 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.861782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.862595 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.862596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.863463 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.863466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.863923 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.863924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.863971 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.864376 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.864381 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.866080 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 03:48:18.866305 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:18.861470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:39097 (local address 127.7.132.190:34061)
0812 03:48:18.861500 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.861502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.861509 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:48:18.861526 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.861527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.861534 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.861588 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.861842 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.862511 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.863566 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.863847 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.863916 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.866176 (+ 2260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:39097: BlockingRecv error: recv got EOF from 127.7.132.129:39097 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 03:48:18.866308 22075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:48:18.929288 22081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:18.929464 22081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:18.929484 22081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:18.930546 22081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:18.930585 22081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:18.930610 22081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:18.932006 22081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.130.keytab
--principal=oryx/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:35457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=33607
--tserver_master_addrs=127.7.132.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:18.932184 22081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:18.932355 22081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:18.937039 22096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.937047 22092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:18.937202 22094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:18.937343 22081 server_base.cc:1034] running on GCE node
I20260812 03:48:18.937561 22081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:18.937799 22081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:18.938961 22081 hybrid_clock.cc:648] HybridClock initialized: now 1786506498938954 us; error 41 us; skew 500 ppm
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506498, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:18.941350 22081 init.cc:377] Logged in from keytab as oryx/127.7.132.130@KRBTEST.COM (short username oryx)
I20260812 03:48:18.941762 22081 webserver.cc:460] Webserver started at http://127.7.132.130:33607/ using document root <none> and password file <none>
I20260812 03:48:18.941922 22081 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:18.941972 22081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:18.943265 22081 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:18.943796 22103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.943977 22081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:18.944063 22081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "ed42b3d4a3cb41bb991c53c85daf3edb"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "b35d30421e965c2b19d53287aa197110"
server_key_iv: "e586c6f2f7c2a106f79ea5ffe5275128"
server_key_version: "encryptionkey@0"
I20260812 03:48:18.944348 22081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:18.969269 22081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:18.971160 22081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:18.971271 22081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:18.971457 22081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:18.971774 22081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:18.971808 22081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.971853 22081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:18.971879 22081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:18.977416 22081 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:35457
I20260812 03:48:18.977499 22277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:35457 every 8 connection(s)
I20260812 03:48:18.977825 22081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:18.980412 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 22081
I20260812 03:48:18.980702 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:39755
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:34061
--webserver_port=42363
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.132.131
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:48:18.985211 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:18.978535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:37773 (local address 127.7.132.190:34061)
0812 03:48:18.978603 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.978605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.978896 (+ 291us) server_negotiation.cc:407] Connection header received
0812 03:48:18.979431 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.979432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.979447 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.979495 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.980423 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.981020 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.981966 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.982152 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.982228 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.985156 (+ 2928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:37773: BlockingRecv error: recv got EOF from 127.7.132.130:37773 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:48:18.985280 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.978521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:37773)
0812 03:48:18.978763 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.978813 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.979323 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.979575 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.979579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.980139 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.980320 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.980325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.981107 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.981108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.981874 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.981876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.982243 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.982244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.982387 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.982888 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.982896 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.985083 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":119,"threads_started":1}
W20260812 03:48:18.985491 22278 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:34061 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.132.190:34061: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:18.989634 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.985720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:36401)
0812 03:48:18.985758 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.985766 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.985812 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.985940 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.985941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.986059 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.986093 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.986095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.986653 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.986654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.987165 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.987166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.987547 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.987548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.987614 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.987998 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.988003 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.989560 (+ 1557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 03:48:18.989686 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:18.985776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:36401 (local address 127.7.132.190:34061)
0812 03:48:18.985830 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.985833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.985839 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:48:18.985861 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.985862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.985870 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.985936 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.986162 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.986563 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.987274 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.987467 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.987546 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.989627 (+ 2081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:36401: BlockingRecv error: recv got EOF from 127.7.132.130:36401 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:18 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:18.993793 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:18.989846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:41335)
0812 03:48:18.989971 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:18.989980 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:18.990022 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:18.990139 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:18.990140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:18.990262 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:18.990283 (+ 21us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.990284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.990881 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.990882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.991345 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:18.991346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.991691 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.991692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:18.991742 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.992155 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:18.992160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:18.993734 (+ 1574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
W20260812 03:48:18.993870 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:18.989881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:41335 (local address 127.7.132.190:34061)
0812 03:48:18.989913 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:18.989915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:18.990011 (+ 96us) server_negotiation.cc:407] Connection header received
0812 03:48:18.990076 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:18.990077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:18.990085 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:18.990127 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:18.990372 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.990813 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.991439 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:18.991622 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:18.991692 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:18.993807 (+ 2115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:41335: BlockingRecv error: recv got EOF from 127.7.132.130:41335 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 03:48:18.993911 22278 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:48:19.061836 22281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:19.062052 22281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:19.062093 22281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:19.063282 22281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:19.063347 22281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:19.063392 22281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:19.064850 22281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/krb5kdc/oryx_127.7.132.131.keytab
--principal=oryx/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:39755
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=42363
--tserver_master_addrs=127.7.132.190:34061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:19.065106 22281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:19.065328 22281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:19.070621 22295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:19.070617 22299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:19.070669 22293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:19.070880 22281 server_base.cc:1034] running on GCE node
I20260812 03:48:19.071107 22281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:19.071362 22281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:19.072525 22281 hybrid_clock.cc:648] HybridClock initialized: now 1786506499072512 us; error 30 us; skew 500 ppm
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506499, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:19.075001 22281 init.cc:377] Logged in from keytab as oryx/127.7.132.131@KRBTEST.COM (short username oryx)
I20260812 03:48:19.075428 22281 webserver.cc:460] Webserver started at http://127.7.132.131:42363/ using document root <none> and password file <none>
I20260812 03:48:19.075662 22281 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:19.075744 22281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:19.077014 22281 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:19.077579 22311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:19.077751 22281 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:19.077848 22281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "be1c0d83fec44c468a4b8661f6bc8ed2"
format_stamp: "Formatted at 2026-08-12 03:48:18 on dist-test-slave-39l8"
server_key: "6017ef437f0356d011c1b36da6032d18"
server_key_iv: "ad59e161ad2fb13d7223cce6e0bfee0f"
server_key_version: "encryptionkey@0"
I20260812 03:48:19.078210 22281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:19.090093 22281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:19.091794 22281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:19.091946 22281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:19.092168 22281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:19.092476 22281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:19.092532 22281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:19.092572 22281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:19.092602 22281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:19.098446 22281 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:39755
I20260812 03:48:19.098493 22480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:39755 every 8 connection(s)
I20260812 03:48:19.098910 22281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:19.106395 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 22281
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:48:19.110879 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:19.099624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48879 (local address 127.7.132.190:34061)
0812 03:48:19.099665 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:19.099668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:19.099945 (+ 277us) server_negotiation.cc:407] Connection header received
0812 03:48:19.100520 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:19.100521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:19.100533 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:19.100587 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:19.101470 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.101962 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.103006 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.103216 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.103294 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.110798 (+ 7504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:48879: BlockingRecv error: recv got EOF from 127.7.132.131:48879 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:48:19.110903 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:19.099592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:48879)
0812 03:48:19.099833 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:19.099864 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:19.100417 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:19.100665 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:19.100669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:19.101194 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:19.101382 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.101386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.102056 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.102057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.102920 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.102923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.103301 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.103302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.103433 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.108445 (+ 5012us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:19.108454 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:19.110669 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":166,"threads_started":1}
W20260812 03:48:19.111172 22481 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:34061 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.132.190:34061: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
I20260812 03:48:19.113745 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:19.106916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.7.132.190:34061)
0812 03:48:19.107028 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:19.107029 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:19.107038 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:19.107069 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:19.107070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:19.107078 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:19.107134 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:19.107810 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.108246 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.109049 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.109239 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.109288 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.110512 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:19.110517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:19.110518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:19.110530 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:19.112280 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:19.112659 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:19.112662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:19.112662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:19.112747 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:19.112979 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:19.112980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:19.112980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:19.113200 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:19.113341 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:19.113482 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 03:48:19.113591 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":73,"threads_started":1}
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:19.115784 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:19.111392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:38409)
0812 03:48:19.111648 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:19.111657 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:19.111721 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:19.111848 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:19.111849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:19.111976 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:19.112015 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.112016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.112780 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.112781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.113270 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.113271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.113624 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.113624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.113694 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.114092 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:19.114097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:19.115746 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244}
W20260812 03:48:19.115871 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:19.111442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:38409 (local address 127.7.132.190:34061)
0812 03:48:19.111484 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:19.111486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:19.111707 (+ 221us) server_negotiation.cc:407] Connection header received
0812 03:48:19.111779 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:19.111780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:19.111789 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:19.111843 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:19.112099 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.112704 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.113354 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.113555 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.113625 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.115819 (+ 2194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:38409: BlockingRecv error: recv got EOF from 127.7.132.131:38409 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:19.119261 22483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:19.116041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:55857)
0812 03:48:19.116088 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:19.116097 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:19.116132 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:19.116255 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:19.116255 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:19.116352 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:19.116374 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.116375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.116889 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.116890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.117321 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.117322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.117597 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.117598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.117630 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.117915 (+ 285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:19.117918 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:19.119224 (+ 1306us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:48:19.119354 22083 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:19.116108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:55857 (local address 127.7.132.190:34061)
0812 03:48:19.116163 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:19.116165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:19.116173 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:48:19.116191 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:19.116192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:19.116200 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:19.116253 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:19.116428 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.116826 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.117384 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.117540 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.117585 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.119297 (+ 1712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:55857: BlockingRecv error: recv got EOF from 127.7.132.131:55857 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 03:48:19.119396 22481 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:19.871622 22487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:19.866858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:45093)
0812 03:48:19.867004 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:19.867017 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:19.867081 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:19.867313 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:19.867314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:19.867476 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:19.867523 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.867525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.868282 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.868283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.868877 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.868879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.869196 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.869196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.869234 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.869785 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:19.869790 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:19.871541 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":55,"threads_started":1}
W20260812 03:48:19.871718 22488 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:19.867037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:45093 (local address 127.7.132.190:34061)
0812 03:48:19.867153 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:19.867155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:19.867162 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:48:19.867182 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:19.867182 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:19.867191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:19.867286 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:19.867602 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.868213 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.868964 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.869135 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.869183 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.871647 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:45093: BlockingRecv error: recv got EOF from 127.7.132.129:45093 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:19 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:19.998680 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:19.994356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:39713)
0812 03:48:19.994495 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:19.994506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:19.994577 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:19.994799 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:19.994800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:19.994971 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:19.995017 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.995018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.995618 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.995619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.996125 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:19.996125 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.996406 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.996407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:19.996443 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.996838 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:19.996842 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:19.998617 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
W20260812 03:48:19.998833 22488 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:19.994533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:39713 (local address 127.7.132.190:34061)
0812 03:48:19.994627 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:19.994629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:19.994636 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:48:19.994657 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:19.994658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:19.994669 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:19.994733 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:19.995081 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.995557 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.996187 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:19.996349 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:19.996394 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:19.998756 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:39713: BlockingRecv error: recv got EOF from 127.7.132.130:39713 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 03:48:20 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:20 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:20.124416 22493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:20.119846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:47807)
0812 03:48:20.119981 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:20.119992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:20.120057 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:20.120185 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:20.120186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:20.120347 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:20.120395 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:20.120397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:20.121072 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:20.121074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:20.121628 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:20.121629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:20.121942 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:20.121943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:20.121978 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:20.122467 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:20.122479 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:20.124337 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
W20260812 03:48:20.124532 22488 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:20.119937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:47807 (local address 127.7.132.190:34061)
0812 03:48:20.119998 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:20.120000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:20.120049 (+ 49us) server_negotiation.cc:407] Connection header received
0812 03:48:20.120099 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:20.120099 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:20.120111 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:20.120163 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:20.120456 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:20.120990 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:20.121696 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:20.121883 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:20.121931 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:20.124437 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:47807: BlockingRecv error: recv got EOF from 127.7.132.131:47807 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 03:48:20 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:20 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:20.881075 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:20.872123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:36751)
0812 03:48:20.872258 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:20.872275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:20.872322 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:20.872458 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:20.872459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:20.872612 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:20.872693 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:20.872695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:20.873810 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:20.873812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:20.874341 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:20.874342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:20.874774 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:20.874775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:20.874842 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:20.879021 (+ 4179us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:20.879025 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:20.880984 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
W20260812 03:48:20.881134 22496 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:20.872184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:36751 (local address 127.7.132.190:34061)
0812 03:48:20.872306 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:20.872308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:20.872318 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:20.872378 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:20.872379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:20.872390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:20.872453 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:20.872787 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:20.873712 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:20.874430 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:20.874650 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:20.874771 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:20.881064 (+ 6293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:36751: BlockingRecv error: recv got EOF from 127.7.132.129:36751 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
Aug 12 03:48:20 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:21 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:21.003611 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:20.999176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:60153)
0812 03:48:20.999327 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:20.999339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:20.999405 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:20.999526 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:20.999527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:20.999694 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:20.999744 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:20.999746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.000404 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.000405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:21.000921 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:21.000923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.001278 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.001278 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:21.001332 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:21.001771 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:21.001776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:21.003553 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
W20260812 03:48:21.003746 22496 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:20.999280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:60153 (local address 127.7.132.190:34061)
0812 03:48:20.999332 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:20.999339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:20.999387 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:48:20.999458 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:20.999459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:20.999469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:20.999509 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:20.999808 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.000318 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.001008 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.001198 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.001279 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:21.003659 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:60153: BlockingRecv error: recv got EOF from 127.7.132.130:60153 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 03:48:21 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:21 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:21.129686 22502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:21.124877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:37097)
0812 03:48:21.125041 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:21.125051 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:21.125099 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:21.125311 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:21.125313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:21.125497 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:21.125545 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:21.125547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.126225 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.126226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:21.126818 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:21.126820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.127170 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.127171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:21.127237 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:21.127746 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:21.127754 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:21.129629 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
W20260812 03:48:21.129827 22496 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:21.125106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37097 (local address 127.7.132.190:34061)
0812 03:48:21.125193 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:21.125195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:21.125202 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:48:21.125223 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:21.125223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:21.125234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:21.125301 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:21.125630 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.126122 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.126908 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.127088 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.127164 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:21.129745 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:37097: BlockingRecv error: recv got EOF from 127.7.132.131:37097 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 03:48:21 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:21 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:21.886014 22503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:21.881518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:53665)
0812 03:48:21.881689 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:21.881698 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:21.881751 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:21.881875 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:21.881876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:21.882062 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:21.882112 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:21.882113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.882785 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.882786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:21.883302 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:21.883304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.883623 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.883624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:21.883680 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:21.884169 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:21.884174 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:21.885958 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
W20260812 03:48:21.886128 22504 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:21.881606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:53665 (local address 127.7.132.190:34061)
0812 03:48:21.881757 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:21.881759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:21.881771 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:48:21.881796 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:21.881797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:21.881806 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:21.881860 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:21.882198 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.882713 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.883374 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:21.883561 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:21.883610 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:21.886050 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:53665: BlockingRecv error: recv got EOF from 127.7.132.129:53665 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":88,"threads_started":1}
Aug 12 03:48:22 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:22 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:22.009016 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:22.004111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:39369)
0812 03:48:22.004278 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:22.004287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:22.004367 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:22.004520 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:22.004521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:22.004713 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:22.004760 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:22.004761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.005460 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.005461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:22.006005 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:22.006007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.006397 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.006399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:22.006464 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:22.007008 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:22.007030 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:22.008961 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
W20260812 03:48:22.009182 22504 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:22.004155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:39369 (local address 127.7.132.190:34061)
0812 03:48:22.004225 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:22.004228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:22.004351 (+ 123us) server_negotiation.cc:407] Connection header received
0812 03:48:22.004442 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:22.004443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:22.004457 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:22.004504 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:22.004859 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.005362 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.006098 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.006308 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.006384 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:22.009091 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:39369: BlockingRecv error: recv got EOF from 127.7.132.130:39369 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:48:22 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:22 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:22.134954 22507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:22.130211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:36543)
0812 03:48:22.130402 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:22.130413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:22.130485 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:22.130633 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:22.130635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:22.130834 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:22.130882 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:22.130884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.131523 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.131524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:22.132046 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:22.132048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.132504 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.132505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:22.132585 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:22.133004 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:22.133009 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:22.134887 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":117,"threads_started":1}
W20260812 03:48:22.135038 22504 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:22.130302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:36543 (local address 127.7.132.190:34061)
0812 03:48:22.130393 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:22.130395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:22.130464 (+ 69us) server_negotiation.cc:407] Connection header received
0812 03:48:22.130538 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:22.130539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:22.130553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:22.130624 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:22.130961 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.131440 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.132174 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.132418 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.132499 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:22.134977 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:36543: BlockingRecv error: recv got EOF from 127.7.132.131:36543 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 03:48:22 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:22 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:22.891299 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:22.886510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:52361)
0812 03:48:22.886674 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:22.886685 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:22.886790 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:22.886930 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:22.886931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:22.887096 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:22.887144 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:22.887145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.887766 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.887767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:22.888294 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:22.888295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.888610 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.888611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:22.888658 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:22.889172 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:22.889178 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:22.891176 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
W20260812 03:48:22.891371 22509 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:22.886600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:52361 (local address 127.7.132.190:34061)
0812 03:48:22.886731 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:22.886733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:22.886787 (+ 54us) server_negotiation.cc:407] Connection header received
0812 03:48:22.886842 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:22.886843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:22.886853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:22.886913 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:22.887208 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.887700 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.888362 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:22.888547 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:22.888596 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:22.891271 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:52361: BlockingRecv error: recv got EOF from 127.7.132.129:52361 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":51,"threads_started":1}
Aug 12 03:48:23 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:23 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:23.014060 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:23.009564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:35313)
0812 03:48:23.009741 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:23.009755 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:23.009825 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:23.009982 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:23.009983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:23.010144 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:23.010195 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:23.010196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.010911 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.010912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:23.011414 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:23.011415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.011750 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.011751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:23.011820 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:23.012268 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:23.012273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:23.013968 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
W20260812 03:48:23.014118 22509 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:23.009613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:35313 (local address 127.7.132.190:34061)
0812 03:48:23.009710 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:23.009713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:23.009824 (+ 111us) server_negotiation.cc:407] Connection header received
0812 03:48:23.009884 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:23.009885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:23.009902 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:23.009964 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:23.010259 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.010843 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.011485 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.011670 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.011748 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:23.014058 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:35313: BlockingRecv error: recv got EOF from 127.7.132.130:35313 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 03:48:23 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:23 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:23.139849 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:23.135403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:60429)
0812 03:48:23.135528 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:23.135539 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:23.135616 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:23.135741 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:23.135741 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:23.135908 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:23.135957 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:23.135958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.136567 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.136567 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:23.137030 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:23.137031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.137315 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.137316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:23.137351 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:23.137871 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:23.137876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:23.139794 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
W20260812 03:48:23.139969 22509 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:23.135473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:60429 (local address 127.7.132.190:34061)
0812 03:48:23.135542 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:23.135544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:23.135611 (+ 67us) server_negotiation.cc:407] Connection header received
0812 03:48:23.135666 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:23.135667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:23.135679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:23.135725 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:23.136016 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.136503 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.137092 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.137256 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.137300 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:23.139877 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:60429: BlockingRecv error: recv got EOF from 127.7.132.131:60429 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 03:48:23 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:23 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:23.896508 22514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:23.891957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:57145)
0812 03:48:23.892140 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:23.892152 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:23.892216 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:23.892360 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:23.892360 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:23.892510 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:23.892559 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:23.892560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.893208 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.893209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:23.893758 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:23.893760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.894082 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.894083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:23.894118 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:23.894578 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:23.894583 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:23.896407 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
W20260812 03:48:23.896559 22515 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:23.892047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:57145 (local address 127.7.132.190:34061)
0812 03:48:23.892191 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:23.892193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:23.892203 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:23.892278 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:23.892278 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:23.892289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:23.892346 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:23.892620 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.893144 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.893841 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:23.894025 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:23.894069 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:23.896492 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:57145: BlockingRecv error: recv got EOF from 127.7.132.129:57145 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
Aug 12 03:48:24 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:24 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:24.024716 22516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:24.018757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:47777)
0812 03:48:24.018878 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:24.018889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:24.018946 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:24.019106 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:24.019107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:24.019325 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:24.019384 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:24.019387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.020276 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.020277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:24.021100 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:24.021104 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.021550 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.021551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:24.021614 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:24.022437 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:24.022442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:24.024621 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
W20260812 03:48:24.024761 22515 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:24.018843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:47777 (local address 127.7.132.190:34061)
0812 03:48:24.018919 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:24.018921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:24.018931 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:24.019021 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:24.019022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:24.019035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:24.019085 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:24.019464 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.020191 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.021185 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.021479 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.021550 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:24.024695 (+ 3145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:47777: BlockingRecv error: recv got EOF from 127.7.132.130:47777 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 03:48:24 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:24 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:24.144992 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:24.140304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:35811)
0812 03:48:24.140479 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:24.140490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:24.140573 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:24.140738 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:24.140739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:24.140927 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:24.140974 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:24.140975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.141645 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.141646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:24.142309 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:24.142312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.142720 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.142722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:24.142782 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:24.143219 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:24.143224 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:24.144936 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
W20260812 03:48:24.145087 22515 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:24.140389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:35811 (local address 127.7.132.190:34061)
0812 03:48:24.140458 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:24.140461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:24.140553 (+ 92us) server_negotiation.cc:407] Connection header received
0812 03:48:24.140634 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:24.140635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:24.140665 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:24.140729 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:24.141051 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.141554 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.142398 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.142604 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.142686 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:24.145019 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:35811: BlockingRecv error: recv got EOF from 127.7.132.131:35811 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 03:48:24 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:24 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:24.901719 22519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:24.896983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:41815)
0812 03:48:24.897128 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:24.897137 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:24.897184 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:24.897338 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:24.897339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:24.897506 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:24.897553 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:24.897555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.898164 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.898165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:24.898744 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:24.898748 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.899244 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.899245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:24.899315 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:24.899834 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:24.899838 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:24.901663 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
W20260812 03:48:24.901881 22520 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:24.897055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:41815 (local address 127.7.132.190:34061)
0812 03:48:24.897201 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:24.897203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:24.897210 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:48:24.897238 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:24.897239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:24.897248 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:24.897322 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:24.897616 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.898101 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.898824 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:24.899159 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:24.899241 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:24.901782 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:41815: BlockingRecv error: recv got EOF from 127.7.132.129:41815 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":103,"threads_started":1}
Aug 12 03:48:25 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:25 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:25.029937 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:25.025413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:50411)
0812 03:48:25.025550 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:25.025560 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:25.025642 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:25.025794 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:25.025795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:25.025967 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:25.026011 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:25.026013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.026627 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.026628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:25.027179 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:25.027182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.027534 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.027535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:25.027590 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:25.028061 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:25.028066 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:25.029847 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
W20260812 03:48:25.029986 22520 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:25.025460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:50411 (local address 127.7.132.190:34061)
0812 03:48:25.025521 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:25.025524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:25.025620 (+ 96us) server_negotiation.cc:407] Connection header received
0812 03:48:25.025703 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:25.025704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:25.025722 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:25.025785 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:25.026089 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.026542 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.027262 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.027439 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.027533 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:25.029924 (+ 2391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:50411: BlockingRecv error: recv got EOF from 127.7.132.130:50411 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 03:48:25 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:25 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:25.150198 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:25.145550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:49819)
0812 03:48:25.145747 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:25.145756 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:25.145831 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:25.145974 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:25.145975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:25.146147 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:25.146194 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:25.146196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.146956 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.146957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:25.147488 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:25.147490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.147855 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.147856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:25.147921 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:25.148326 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:25.148330 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:25.150104 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":115,"threads_started":1}
W20260812 03:48:25.150234 22520 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:25.145631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49819 (local address 127.7.132.190:34061)
0812 03:48:25.145707 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:25.145709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:25.145810 (+ 101us) server_negotiation.cc:407] Connection header received
0812 03:48:25.145890 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:25.145890 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:25.145904 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:25.145968 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:25.146304 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.146877 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.147573 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.147768 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.147845 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:25.150175 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:49819: BlockingRecv error: recv got EOF from 127.7.132.131:49819 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 03:48:25 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:25 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:25.911957 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:25.902240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:55273)
0812 03:48:25.902399 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:25.902409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:25.902466 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:25.902641 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:25.902643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:25.902865 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:25.902912 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:25.902914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.903587 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.903589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:25.904124 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:25.904126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.904496 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.904497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:25.904561 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:25.910052 (+ 5491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:25.910057 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:25.911898 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
W20260812 03:48:25.912118 22524 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:25.902317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:55273 (local address 127.7.132.190:34061)
0812 03:48:25.902418 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:25.902419 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:25.902451 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:48:25.902527 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:25.902528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:25.902539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:25.902630 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:25.903002 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.903502 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.904223 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:25.904421 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:25.904496 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:25.912015 (+ 7519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:55273: BlockingRecv error: recv got EOF from 127.7.132.129:55273 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":48,"threads_started":1}
Aug 12 03:48:26 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:26 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:26.035010 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:26.030468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:34925)
0812 03:48:26.030638 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:26.030663 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:26.030778 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:26.030928 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:26.030928 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:26.031080 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:26.031126 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:26.031128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.031721 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.031722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:26.032258 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:26.032260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.032654 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.032655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:26.032697 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:26.033116 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:26.033121 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:26.034955 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
W20260812 03:48:26.035100 22524 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:26.030551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:34925 (local address 127.7.132.190:34061)
0812 03:48:26.030622 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:26.030624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:26.030738 (+ 114us) server_negotiation.cc:407] Connection header received
0812 03:48:26.030846 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:26.030847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:26.030857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:26.030910 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:26.031185 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.031659 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.032350 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.032589 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.032637 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:26.035038 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:34925: BlockingRecv error: recv got EOF from 127.7.132.130:34925 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 03:48:26 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:26 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:26.155530 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:26.150745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:33763)
0812 03:48:26.150930 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:26.150941 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:26.151033 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:26.151198 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:26.151199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:26.151373 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:26.151438 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:26.151440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.152108 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.152110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:26.152622 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:26.152623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.153002 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.153003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:26.153064 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:26.153495 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:26.153501 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:26.155434 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":104,"threads_started":1}
W20260812 03:48:26.155566 22524 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:26.150837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33763 (local address 127.7.132.190:34061)
0812 03:48:26.150911 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:26.150913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:26.151011 (+ 98us) server_negotiation.cc:407] Connection header received
0812 03:48:26.151099 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:26.151100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:26.151114 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:26.151191 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:26.151518 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.152016 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.152717 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.152904 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.153005 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:26.155505 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:33763: BlockingRecv error: recv got EOF from 127.7.132.131:33763 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 03:48:26 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:26 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:26.916705 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:26.912401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:56587)
0812 03:48:26.912564 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:26.912574 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:26.912624 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:26.912771 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:26.912772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:26.912944 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:26.912993 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:26.912994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.913628 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.913629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:26.914139 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:26.914140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.914497 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.914497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:26.914536 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:26.914993 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:26.914997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:26.916613 (+ 1616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
W20260812 03:48:26.916759 22530 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:26.912514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:56587 (local address 127.7.132.190:34061)
0812 03:48:26.912650 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:26.912652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:26.912661 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:26.912686 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:26.912687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:26.912697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:26.912753 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:26.913061 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.913563 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.914209 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:26.914418 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:26.914482 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:26.916690 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:56587: BlockingRecv error: recv got EOF from 127.7.132.129:56587 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
Aug 12 03:48:27 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:27 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:27.040172 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:27.035471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:58823)
0812 03:48:27.035603 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:27.035613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:27.035667 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:27.035798 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:27.035799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:27.035957 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:27.036024 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:27.036026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.036776 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.036777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:27.037293 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:27.037294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.037611 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.037612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:27.037662 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:27.038256 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:27.038261 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:27.040063 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
W20260812 03:48:27.040223 22530 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:27.035544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:58823 (local address 127.7.132.190:34061)
0812 03:48:27.035608 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:27.035610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:27.035667 (+ 57us) server_negotiation.cc:407] Connection header received
0812 03:48:27.035725 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:27.035726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:27.035737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:27.035780 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:27.036084 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.036712 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.037365 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.037546 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.037594 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:27.040159 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:58823: BlockingRecv error: recv got EOF from 127.7.132.130:58823 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 03:48:27 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:27 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:27.160671 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:27.156163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:48405)
0812 03:48:27.156347 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:27.156359 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:27.156421 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:27.156558 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:27.156560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:27.156744 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:27.156793 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:27.156795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.157400 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.157400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:27.157905 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:27.157906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.158200 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.158201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:27.158240 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:27.158781 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:27.158786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:27.160565 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
W20260812 03:48:27.160697 22530 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:27.156183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48405 (local address 127.7.132.190:34061)
0812 03:48:27.156253 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:27.156257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:27.156426 (+ 169us) server_negotiation.cc:407] Connection header received
0812 03:48:27.156469 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:27.156470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:27.156484 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:27.156546 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:27.156860 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.157333 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.157971 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.158139 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.158185 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:27.160634 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:48405: BlockingRecv error: recv got EOF from 127.7.132.131:48405 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 03:48:27 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:27 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:27.921902 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:27.917445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:56333)
0812 03:48:27.917598 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:27.917609 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:27.917688 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:27.917826 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:27.917827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:27.917993 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:27.918042 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:27.918043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.918624 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.918624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:27.919111 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:27.919112 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.919410 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.919411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:27.919449 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:27.919864 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:27.919869 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:27.921809 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":68,"threads_started":1}
W20260812 03:48:27.921952 22534 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:27.917503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:56333 (local address 127.7.132.190:34061)
0812 03:48:27.917671 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:27.917673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:27.917681 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:48:27.917744 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:27.917744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:27.917755 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:27.917804 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:27.918104 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.918565 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.919177 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:27.919350 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:27.919400 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:27.921883 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:56333: BlockingRecv error: recv got EOF from 127.7.132.129:56333 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
Aug 12 03:48:28 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:28 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:28.045045 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:28.040741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:54997)
0812 03:48:28.040887 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:28.040897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:28.040957 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:28.041069 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:28.041070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:28.041265 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:28.041313 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:28.041314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.041871 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.041871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:28.042336 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:28.042337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.042617 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.042618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:28.042654 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:28.043160 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:28.043165 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:28.044989 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
W20260812 03:48:28.045135 22534 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:28.040783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:54997 (local address 127.7.132.190:34061)
0812 03:48:28.040844 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:28.040846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:28.040955 (+ 109us) server_negotiation.cc:407] Connection header received
0812 03:48:28.040997 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:28.040998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:28.041009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:28.041054 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:28.041368 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.041812 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.042398 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.042561 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.042606 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:28.045072 (+ 2466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:54997: BlockingRecv error: recv got EOF from 127.7.132.130:54997 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 03:48:28 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:28 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:28.165881 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:28.161140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:36617)
0812 03:48:28.161320 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:28.161337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:28.161418 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:28.161610 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:28.161611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:28.161798 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:28.161845 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:28.161847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.162500 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.162501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:28.163033 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:28.163035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.163436 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.163437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:28.163506 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:28.163947 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:28.163952 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:28.165826 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":105,"threads_started":1}
W20260812 03:48:28.166013 22534 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:28.161229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:36617 (local address 127.7.132.190:34061)
0812 03:48:28.161304 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:28.161307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:28.161397 (+ 90us) server_negotiation.cc:407] Connection header received
0812 03:48:28.161488 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:28.161489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:28.161521 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:28.161603 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:28.161931 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.162416 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.163161 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.163362 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.163438 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:28.165934 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:36617: BlockingRecv error: recv got EOF from 127.7.132.131:36617 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 03:48:28 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:28 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:28.926762 22538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:28.922447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:45359)
0812 03:48:28.922567 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:28.922579 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:28.922644 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:28.922841 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:28.922842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:28.923015 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:28.923063 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:28.923065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.923723 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.923723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:28.924237 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:28.924239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.924559 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.924559 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:28.924594 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:28.925033 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:28.925037 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:28.926650 (+ 1613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
W20260812 03:48:28.926878 22539 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:28.922515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:45359 (local address 127.7.132.190:34061)
0812 03:48:28.922642 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:28.922644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:28.922661 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:48:28.922742 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:28.922743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:28.922760 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:28.922815 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:28.923132 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.923647 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.924319 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:28.924503 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:28.924547 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:28.926804 (+ 2257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:45359: BlockingRecv error: recv got EOF from 127.7.132.129:45359 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
Aug 12 03:48:29 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:29 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:29.050418 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:29.045748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:58385)
0812 03:48:29.045917 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:29.045928 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:29.045996 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:29.046142 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:29.046143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:29.046327 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:29.046373 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:29.046376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.047105 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.047106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:29.047613 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:29.047614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.048001 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.048002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:29.048057 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:29.048489 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:29.048493 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:29.050361 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
W20260812 03:48:29.050554 22539 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:29.045845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:58385 (local address 127.7.132.190:34061)
0812 03:48:29.045914 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:29.045917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:29.045974 (+ 57us) server_negotiation.cc:407] Connection header received
0812 03:48:29.046057 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:29.046058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:29.046075 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:29.046135 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:29.046452 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.047023 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.047700 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.047909 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.048003 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:29.050444 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:58385: BlockingRecv error: recv got EOF from 127.7.132.130:58385 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 03:48:29 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:29 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:29.171185 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:29.166518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:52887)
0812 03:48:29.166694 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:29.166735 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:29.166801 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:29.166930 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:29.166930 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:29.167111 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:29.167160 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:29.167162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.167779 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.167780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:29.168289 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:29.168290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.168683 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.168684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:29.168744 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:29.169281 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:29.169286 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:29.171118 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
W20260812 03:48:29.171309 22539 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:29.166659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:52887 (local address 127.7.132.190:34061)
0812 03:48:29.166757 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:29.166759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:29.166789 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:48:29.166845 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:29.166846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:29.166857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:29.166912 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:29.167239 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.167708 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.168377 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.168571 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.168668 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:29.171213 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:52887: BlockingRecv error: recv got EOF from 127.7.132.131:52887 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 03:48:29 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:29 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:29.931326 22543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:29.927193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:38297)
0812 03:48:29.927356 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:29.927367 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:29.927428 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:29.927566 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:29.927567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:29.927750 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:29.927797 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:29.927798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.928367 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.928367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:29.928842 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:29.928843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.929130 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.929131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:29.929166 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:29.929566 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:29.929571 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:29.931269 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
W20260812 03:48:29.931444 22544 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:29.927271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:38297 (local address 127.7.132.190:34061)
0812 03:48:29.927385 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:29.927387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:29.927413 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:48:29.927482 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:29.927483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:29.927493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:29.927548 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:29.927857 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.928308 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.928906 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:29.929074 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:29.929119 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:29.931361 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:38297: BlockingRecv error: recv got EOF from 127.7.132.129:38297 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
Aug 12 03:48:30 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:30 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:30.055732 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:30.050977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:40817)
0812 03:48:30.051166 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:30.051184 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:30.051280 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:30.051438 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:30.051439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:30.051623 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:30.051667 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:30.051669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.052368 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.052370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:30.052935 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:30.052936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.053324 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.053325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:30.053392 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:30.053909 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:30.053914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:30.055667 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
W20260812 03:48:30.055832 22544 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:30.051179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:40817 (local address 127.7.132.190:34061)
0812 03:48:30.051246 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:30.051248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:30.051258 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:30.051337 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:30.051338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:30.051351 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:30.051416 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:30.051751 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.052282 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.053021 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.053236 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.053311 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:30.055747 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:40817: BlockingRecv error: recv got EOF from 127.7.132.130:40817 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 03:48:30 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:30 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:30.176527 22546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:30.171656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:43491)
0812 03:48:30.171821 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:30.171835 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:30.171934 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:30.172125 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:30.172126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:30.172399 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:30.172460 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:30.172463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.173215 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.173216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:30.173752 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:30.173753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.174063 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.174064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:30.174102 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:30.174559 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:30.174564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:30.176421 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
W20260812 03:48:30.176564 22544 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:30.171851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:43491 (local address 127.7.132.190:34061)
0812 03:48:30.171917 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:30.171919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:30.171928 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:30.171991 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:30.171992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:30.172006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:30.172116 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:30.172554 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.173139 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.173823 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.174000 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.174049 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:30.176500 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:43491: BlockingRecv error: recv got EOF from 127.7.132.131:43491 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 03:48:30 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:30 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:30.942616 22548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:30.931836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:53403)
0812 03:48:30.932037 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:30.932050 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:30.932159 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:30.932364 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:30.932365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:30.932609 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:30.932663 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:30.932665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.933396 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.933397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:30.933990 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:30.933992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.934362 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.934363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:30.934422 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:30.940879 (+ 6457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:30.940884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:30.942509 (+ 1625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
W20260812 03:48:30.942668 22549 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:30.931902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:53403 (local address 127.7.132.190:34061)
0812 03:48:30.932056 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:30.932058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:30.932186 (+ 128us) server_negotiation.cc:407] Connection header received
0812 03:48:30.932259 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:30.932260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:30.932275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:30.932358 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:30.932761 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.933281 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.934085 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:30.934288 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:30.934364 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:30.942587 (+ 8223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:53403: BlockingRecv error: recv got EOF from 127.7.132.129:53403 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
Aug 12 03:48:31 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:31 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:31.060425 22550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:31.056146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:44195)
0812 03:48:31.056317 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:31.056326 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:31.056378 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:31.056517 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:31.056518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:31.056685 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:31.056731 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:31.056733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.057339 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.057340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:31.057832 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:31.057833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.058136 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.058137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:31.058175 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:31.058608 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:31.058613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:31.060368 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
W20260812 03:48:31.060559 22549 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:31.056245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:44195 (local address 127.7.132.190:34061)
0812 03:48:31.056310 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:31.056312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:31.056364 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:48:31.056446 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:31.056447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:31.056458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:31.056497 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:31.056795 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.057273 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.057900 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.058076 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.058124 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:31.060480 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:44195: BlockingRecv error: recv got EOF from 127.7.132.130:44195 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 03:48:31 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:31 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:31.181666 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:31.177084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:35455)
0812 03:48:31.177247 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:31.177258 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:31.177344 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:31.177477 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:31.177477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:31.177646 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:31.177694 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:31.177695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.178337 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.178338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:31.178911 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:31.178913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.179225 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.179226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:31.179269 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:31.179716 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:31.179720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:31.181560 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
W20260812 03:48:31.181710 22549 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:31.177161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:35455 (local address 127.7.132.190:34061)
0812 03:48:31.177209 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:31.177211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:31.177347 (+ 136us) server_negotiation.cc:407] Connection header received
0812 03:48:31.177399 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:31.177399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:31.177411 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:31.177453 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:31.177760 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.178254 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.178985 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.179163 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.179213 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:31.181645 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:35455: BlockingRecv error: recv got EOF from 127.7.132.131:35455 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 03:48:31 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:31 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:31.947919 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:31.943174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:52011)
0812 03:48:31.943364 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:31.943375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:31.943455 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:31.943587 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:31.943588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:31.943770 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:31.943820 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:31.943821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.944443 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.944444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:31.944984 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:31.944986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.945289 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.945290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:31.945327 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:31.945852 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:31.945856 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:31.947796 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":121,"threads_started":1}
W20260812 03:48:31.948020 22554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:31.943273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:52011 (local address 127.7.132.190:34061)
0812 03:48:31.943375 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:31.943377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:31.943445 (+ 68us) server_negotiation.cc:407] Connection header received
0812 03:48:31.943498 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:31.943499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:31.943510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:31.943568 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:31.943887 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.944379 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.945050 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:31.945230 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:31.945278 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:31.947917 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:52011: BlockingRecv error: recv got EOF from 127.7.132.129:52011 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":49,"threads_started":1}
Aug 12 03:48:32 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:32 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:32.065447 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:32.060910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:57475)
0812 03:48:32.061094 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:32.061109 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:32.061181 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:32.061348 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:32.061349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:32.061515 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:32.061561 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:32.061562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.062165 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.062165 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:32.062784 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:32.062787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.063129 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.063129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:32.063175 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:32.063697 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:32.063702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:32.065389 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
W20260812 03:48:32.065546 22554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:32.061146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:57475 (local address 127.7.132.190:34061)
0812 03:48:32.061224 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:32.061226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:32.061241 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:48:32.061265 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:32.061266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:32.061276 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:32.061328 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:32.061628 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.062104 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.062873 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.063061 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.063116 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:32.065480 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:57475: BlockingRecv error: recv got EOF from 127.7.132.130:57475 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 03:48:32 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:32 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:32.186983 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:32.182139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:54681)
0812 03:48:32.182282 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:32.182293 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:32.182398 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:32.182579 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:32.182580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:32.182819 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:32.182866 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:32.182867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.183551 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.183553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:32.184102 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:32.184105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.184472 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.184473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:32.184536 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:32.185086 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:32.185090 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:32.186919 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
W20260812 03:48:32.187076 22554 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:32.182195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:54681 (local address 127.7.132.190:34061)
0812 03:48:32.182266 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:32.182268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:32.182364 (+ 96us) server_negotiation.cc:407] Connection header received
0812 03:48:32.182471 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:32.182472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:32.182487 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:32.182554 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:32.182949 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.183464 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.184189 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.184383 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.184469 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:32.187013 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:54681: BlockingRecv error: recv got EOF from 127.7.132.131:54681 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 03:48:32 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:32 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:32.952920 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:32.948332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:49139)
0812 03:48:32.948497 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:32.948510 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:32.948581 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:32.948730 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:32.948731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:32.948894 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:32.948941 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:32.948942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.949577 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.949577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:32.950134 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:32.950136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.950460 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.950461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:32.950500 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:32.951043 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:32.951048 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:32.952862 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
W20260812 03:48:32.953040 22560 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:32.948415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:49139 (local address 127.7.132.190:34061)
0812 03:48:32.948524 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:32.948526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:32.948562 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:48:32.948642 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:32.948642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:32.948652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:32.948710 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:32.949002 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.949489 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.950208 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:32.950398 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:32.950445 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:32.952955 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:49139: BlockingRecv error: recv got EOF from 127.7.132.129:49139 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
Aug 12 03:48:33 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:33 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:33.070202 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:33.066000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:42131)
0812 03:48:33.066154 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:33.066163 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:33.066218 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:33.066349 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:33.066349 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:33.066533 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:33.066590 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:33.066592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.067179 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.067180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:33.067678 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:33.067680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.067969 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.067969 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:33.068006 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:33.068522 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:33.068526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:33.070136 (+ 1610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260812 03:48:33.070312 22560 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:33.066075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:42131 (local address 127.7.132.190:34061)
0812 03:48:33.066120 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:33.066121 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:33.066218 (+ 97us) server_negotiation.cc:407] Connection header received
0812 03:48:33.066272 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:33.066272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:33.066283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:33.066329 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:33.066647 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.067117 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.067746 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.067911 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.067957 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:33.070240 (+ 2283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:42131: BlockingRecv error: recv got EOF from 127.7.132.130:42131 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 03:48:33 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:33 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.131@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:33.191635 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:33.187506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.131:40425)
0812 03:48:33.187658 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:33.187667 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:33.187725 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:33.187854 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:33.187854 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:33.188015 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:33.188060 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:33.188061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.188625 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.188625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:33.189098 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:33.189099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.189384 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.189384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:33.189419 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:33.189808 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:33.189812 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:33.191577 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
W20260812 03:48:33.191707 22560 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:33.187568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:40425 (local address 127.7.132.190:34061)
0812 03:48:33.187657 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:33.187659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:33.187720 (+ 61us) server_negotiation.cc:407] Connection header received
0812 03:48:33.187783 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:33.187783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:33.187796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:33.187833 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:33.188120 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.188563 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.189168 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.189327 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.189371 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:33.191662 (+ 2291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.131:40425: BlockingRecv error: recv got EOF from 127.7.132.131:40425 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 03:48:33 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:33 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.129@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:33.958206 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:33.953648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.129:55099)
0812 03:48:33.953813 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:33.953823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:33.953880 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:33.954013 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:33.954014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:33.954183 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:33.954234 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:33.954236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.954951 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.954953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:33.955473 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:33.955475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.955849 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.955850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:33.955910 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:33.956376 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:33.956380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:33.958110 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
W20260812 03:48:33.958264 22564 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:33.953749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:55099 (local address 127.7.132.190:34061)
0812 03:48:33.953876 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:33.953878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:33.953887 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:33.953938 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:33.953938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:33.953948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:33.953998 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:33.954302 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.954867 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.955572 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:33.955766 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:33.955832 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:33.958193 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.129:55099: BlockingRecv error: recv got EOF from 127.7.132.129:55099 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":53,"threads_started":1}
Aug 12 03:48:34 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:48:34 dist-test-slave-39l8 krb5kdc[20949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.132.130@KRBTEST.COM for oryx/127.7.132.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:48:34.075582 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.070988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34061 (local address 127.7.132.130:43695)
0812 03:48:34.071171 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:34.071181 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:34.071243 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:34.071367 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:34.071367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:34.071518 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:34.071565 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:34.071567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.072233 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.072234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:34.072812 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:34.072816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.073279 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.073280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:34.073337 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.073791 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:34.073796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:34.075527 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.132.190:34061: Server oryx/127.7.132.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":115,"threads_started":1}
W20260812 03:48:34.075706 22564 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:34.071042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:43695 (local address 127.7.132.190:34061)
0812 03:48:34.071111 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:34.071113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:34.071241 (+ 128us) server_negotiation.cc:407] Connection header received
0812 03:48:34.071299 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:34.071300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:34.071310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:34.071349 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:34.071660 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.072154 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.072912 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.073191 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.073272 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.075626 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.132.130:43695: BlockingRecv error: recv got EOF from 127.7.132.130:43695 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:48:34.115530 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 21861
I20260812 03:48:34.127354 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 22081
I20260812 03:48:34.136384 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 22281
I20260812 03:48:34.142971 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 21758
2026-08-12T03:48:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17645 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T03:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:34Z Disabled control of system clock
I20260812 03:48:34.174340 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41841
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:41841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:48:34.253149 22569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:34.253353 22569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:34.253386 22569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:34.254565 22569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:34.254606 22569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:34.254619 22569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:34.254632 22569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:34.254685 22569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:34.256084 22569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:41841
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:34.256348 22569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:34.256680 22569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:34.261698 22579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:34.261714 22569 server_base.cc:1034] running on GCE node
W20260812 03:48:34.261720 22581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.261685 22578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:34.262149 22569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:34.262455 22569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:34.263628 22569 hybrid_clock.cc:648] HybridClock initialized: now 1786506514263606 us; error 34 us; skew 500 ppm
I20260812 03:48:34.264828 22569 webserver.cc:460] Webserver started at http://127.7.132.190:45535/ using document root <none> and password file <none>
I20260812 03:48:34.265110 22569 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:34.265180 22569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:34.265328 22569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:34.266312 22569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "a304165cf39c462888c606bb284c4c79"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "57bac30846f984d4a956c4d170e448bd"
server_key_iv: "71160dc88bca7bc3754a1a3f05062a6d"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.266671 22569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "a304165cf39c462888c606bb284c4c79"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "57bac30846f984d4a956c4d170e448bd"
server_key_iv: "71160dc88bca7bc3754a1a3f05062a6d"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.267982 22569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:48:34.268793 22598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.268957 22569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:34.269063 22569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "a304165cf39c462888c606bb284c4c79"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "57bac30846f984d4a956c4d170e448bd"
server_key_iv: "71160dc88bca7bc3754a1a3f05062a6d"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.269141 22569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:34.275743 22569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:34.276031 22569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:34.276177 22569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:34.279961 22569 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:41841
I20260812 03:48:34.280030 22666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:41841 every 8 connection(s)
I20260812 03:48:34.280337 22569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:34.281064 22667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:34.283573 22667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79: Bootstrap starting.
I20260812 03:48:34.284207 22667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:34.284507 22667 log.cc:824] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:34.285172 22667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79: No bootstrap required, opened a new log
I20260812 03:48:34.286594 22667 raft_consensus.cc:348] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a304165cf39c462888c606bb284c4c79" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41841 } }
I20260812 03:48:34.286774 22667 raft_consensus.cc:374] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:34.286827 22667 raft_consensus.cc:729] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a304165cf39c462888c606bb284c4c79, State: Initialized, Role: FOLLOWER
I20260812 03:48:34.286962 22667 consensus_queue.cc:260] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a304165cf39c462888c606bb284c4c79" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41841 } }
I20260812 03:48:34.287019 22667 raft_consensus.cc:388] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:34.287038 22667 raft_consensus.cc:482] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:34.287101 22667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:34.287563 22667 raft_consensus.cc:504] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a304165cf39c462888c606bb284c4c79" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41841 } }
I20260812 03:48:34.287667 22667 leader_election.cc:302] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a304165cf39c462888c606bb284c4c79; no voters:
I20260812 03:48:34.287832 22667 leader_election.cc:288] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:34.287910 22671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:34.288117 22667 sys_catalog.cc:564] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:34.288102 22671 raft_consensus.cc:686] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [term 1 LEADER]: Becoming Leader. State: Replica: a304165cf39c462888c606bb284c4c79, State: Running, Role: LEADER
I20260812 03:48:34.288317 22671 consensus_queue.cc:237] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a304165cf39c462888c606bb284c4c79" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41841 } }
I20260812 03:48:34.288616 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 22569
I20260812 03:48:34.288719 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:34.288734 22673 sys_catalog.cc:455] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a304165cf39c462888c606bb284c4c79. Latest consensus state: current_term: 1 leader_uuid: "a304165cf39c462888c606bb284c4c79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a304165cf39c462888c606bb284c4c79" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41841 } } }
I20260812 03:48:34.288830 22673 sys_catalog.cc:458] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:34.288856 22672 sys_catalog.cc:455] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a304165cf39c462888c606bb284c4c79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a304165cf39c462888c606bb284c4c79" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41841 } } }
I20260812 03:48:34.288928 22672 sys_catalog.cc:458] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:34.288935 7698 external_mini_cluster.cc:1428] Setting key 7d90e9226cd3aefe837ceefb5ace6297
I20260812 03:48:34.289057 22682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:34.289553 22682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:34.294763 22682 catalog_manager.cc:1347] Generated new cluster ID: 807ee7de447547ec9ec4bfbb9fbd9ada
I20260812 03:48:34.294829 22682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:34.294986 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.289414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.7.132.190:41841)
0812 03:48:34.289991 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:34.290000 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:34.290030 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:48:34.290267 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:34.290274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:34.290356 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:34.290488 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:34.290786 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.292007 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.292786 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.293401 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.293523 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.294187 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:34.294191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:34.294191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:34.294398 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:34.294438 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:34.294452 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:34.294524 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:34.294613 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 03:48:34.294689 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":86,"threads_started":1}
I20260812 03:48:34.329388 22682 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:34.329942 22682 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:34.335338 22682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a304165cf39c462888c606bb284c4c79: Generated new TSK 0
I20260812 03:48:34.335530 22682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:48:34.344293 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41841
--builtin_ntp_servers=127.7.132.148:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 03:48:34.425845 22704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:34.426059 22704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:34.426090 22704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:34.427191 22704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:34.427248 22704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:34.427287 22704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:34.428609 22704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:34.428875 22704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:34.429158 22704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:34.433974 22716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.433979 22712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.434024 22713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:34.434264 22704 server_base.cc:1034] running on GCE node
I20260812 03:48:34.434608 22704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:34.434882 22704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:34.436043 22704 hybrid_clock.cc:648] HybridClock initialized: now 1786506514436028 us; error 43 us; skew 500 ppm
I20260812 03:48:34.437215 22704 webserver.cc:460] Webserver started at http://127.7.132.129:38267/ using document root <none> and password file <none>
I20260812 03:48:34.437455 22704 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:34.437517 22704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:34.437647 22704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:34.438632 22704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "e1904ea76cd54482b53a17cbc8c3493c"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "19c66ca1475dcba3d373c2f710766346"
server_key_iv: "52a3264af017f625047636b8ca8598a2"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.439033 22704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "e1904ea76cd54482b53a17cbc8c3493c"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "19c66ca1475dcba3d373c2f710766346"
server_key_iv: "52a3264af017f625047636b8ca8598a2"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.440407 22704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:34.441088 22730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.441262 22704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:34.441355 22704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "e1904ea76cd54482b53a17cbc8c3493c"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "19c66ca1475dcba3d373c2f710766346"
server_key_iv: "52a3264af017f625047636b8ca8598a2"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.441438 22704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:34.466717 22704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:34.467065 22704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:34.467209 22704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:34.467424 22704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:34.467759 22704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:34.467813 22704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.467851 22704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:34.467880 22704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.473387 22704 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:36533
I20260812 03:48:34.473454 22900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:36533 every 8 connection(s)
I20260812 03:48:34.473802 22704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:48:34.479120 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.474605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:38615 (local address 127.7.132.190:41841)
0812 03:48:34.474672 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:34.474674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:34.474937 (+ 263us) server_negotiation.cc:407] Connection header received
0812 03:48:34.475527 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:34.475528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:34.475553 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:34.475628 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:34.476324 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.476803 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.477714 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.477897 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.477970 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.478837 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:34.478838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:34.478838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:34.478843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:34.478857 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:34.478861 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:34.478926 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:34.479016 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:48:34.479044 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:48:34.479319 22903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.474560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41841 (local address 127.7.132.129:38615)
0812 03:48:34.474780 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:34.474828 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:34.475397 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:34.475705 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:34.475709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:34.475721 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:34.476213 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:34.476218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.476901 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.476902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:34.477624 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:34.477626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.477979 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.477981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:34.478112 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.478669 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:34.478669 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:34.478696 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:34.478739 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:34.478927 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:34.478934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:34.478937 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:34.479017 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:48:34.479116 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260812 03:48:34.479831 22901 heartbeater.cc:347] Connected to a master server at 127.7.132.190:41841
I20260812 03:48:34.479907 22901 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:34.480098 22901 heartbeater.cc:511] Master 127.7.132.190:41841 requested a full tablet report, sending...
I20260812 03:48:34.480186 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 22704
I20260812 03:48:34.480293 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:34.480548 7698 external_mini_cluster.cc:1428] Setting key 33ec468b6d77e189f959e8dd3a5c496c
I20260812 03:48:34.480847 22624 ts_manager.cc:194] Registered new tserver with Master: e1904ea76cd54482b53a17cbc8c3493c (127.7.132.129:36533)
I20260812 03:48:34.481460 22624 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.129:38615
I20260812 03:48:34.481516 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41841
--builtin_ntp_servers=127.7.132.148:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 03:48:34.560573 22904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:34.560762 22904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:34.560781 22904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:34.561872 22904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:34.561911 22904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:34.561933 22904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:34.563288 22904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:34.563531 22904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:34.563840 22904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:34.568715 22918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.568738 22914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.568778 22911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:34.568902 22904 server_base.cc:1034] running on GCE node
I20260812 03:48:34.569152 22904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:34.569427 22904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:34.570591 22904 hybrid_clock.cc:648] HybridClock initialized: now 1786506514570587 us; error 36 us; skew 500 ppm
I20260812 03:48:34.571875 22904 webserver.cc:460] Webserver started at http://127.7.132.130:33701/ using document root <none> and password file <none>
I20260812 03:48:34.572193 22904 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:34.572265 22904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:34.572403 22904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:34.573544 22904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "2e0a2c09635c46a5bdf1874508e3806a"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "1717796cef666c7429b52bf1ab212d1d"
server_key_iv: "ea94b6c8a038b25ecbc5c611a46954e8"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.573931 22904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "2e0a2c09635c46a5bdf1874508e3806a"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "1717796cef666c7429b52bf1ab212d1d"
server_key_iv: "ea94b6c8a038b25ecbc5c611a46954e8"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.575290 22904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:34.575970 22928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.576134 22904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:34.576234 22904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "2e0a2c09635c46a5bdf1874508e3806a"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "1717796cef666c7429b52bf1ab212d1d"
server_key_iv: "ea94b6c8a038b25ecbc5c611a46954e8"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.576318 22904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:34.587592 22904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:34.587849 22904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:34.587981 22904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:34.588181 22904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:34.588541 22904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:34.588591 22904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.588626 22904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:34.588650 22904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.594274 22904 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:36105
I20260812 03:48:34.594347 23082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:36105 every 8 connection(s)
I20260812 03:48:34.594641 22904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:34.596684 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 22904
I20260812 03:48:34.596763 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:48:34.596961 7698 external_mini_cluster.cc:1428] Setting key 3d3d5346c54c465e039f01db810b0737
I20260812 03:48:34.597986 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41841
--builtin_ntp_servers=127.7.132.148:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:48:34.600359 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.595430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:47597 (local address 127.7.132.190:41841)
0812 03:48:34.595496 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:34.595498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:34.595693 (+ 195us) server_negotiation.cc:407] Connection header received
0812 03:48:34.596176 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:34.596177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:34.596188 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:34.596250 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:34.596964 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.597698 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.598833 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.599123 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.599206 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.600084 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:34.600085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:34.600086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:34.600091 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:34.600105 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:34.600109 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:34.600169 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:34.600255 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 03:48:34.600283 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:48:34.600562 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.595405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41841 (local address 127.7.132.130:47597)
0812 03:48:34.595594 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:34.595625 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:34.596084 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:34.596316 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:34.596319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:34.596326 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:34.596874 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:34.596885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.597795 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.597796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:34.598679 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:34.598681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.599225 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.599226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:34.599339 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.599931 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:34.599931 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:34.599959 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:34.599980 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:34.600169 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:34.600174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:34.600177 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:34.600258 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:48:34.600352 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260812 03:48:34.601125 23083 heartbeater.cc:347] Connected to a master server at 127.7.132.190:41841
I20260812 03:48:34.601210 23083 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:34.601400 23083 heartbeater.cc:511] Master 127.7.132.190:41841 requested a full tablet report, sending...
I20260812 03:48:34.601784 22624 ts_manager.cc:194] Registered new tserver with Master: 2e0a2c09635c46a5bdf1874508e3806a (127.7.132.130:36105)
I20260812 03:48:34.602149 22624 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.130:47597
W20260812 03:48:34.679440 23087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:34.679682 23087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:34.679713 23087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:34.680791 23087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:34.680862 23087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:34.680897 23087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:34.682104 23087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:41841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:34.682333 23087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:34.682595 23087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:34.687335 23098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.687345 23103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.687345 23101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:34.687708 23087 server_base.cc:1034] running on GCE node
I20260812 03:48:34.687937 23087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:34.688166 23087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:34.689319 23087 hybrid_clock.cc:648] HybridClock initialized: now 1786506514689307 us; error 30 us; skew 500 ppm
I20260812 03:48:34.690649 23087 webserver.cc:460] Webserver started at http://127.7.132.131:40283/ using document root <none> and password file <none>
I20260812 03:48:34.690909 23087 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:34.690977 23087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:34.691128 23087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:34.692246 23087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "70befd223ade41fea4a7e017fcdd1dcd"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "7305abc8794f769cd6922440bdcffdfa"
server_key_iv: "ffe52034aa95e2119c2bcc2f1212d2a1"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.692622 23087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "70befd223ade41fea4a7e017fcdd1dcd"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "7305abc8794f769cd6922440bdcffdfa"
server_key_iv: "ffe52034aa95e2119c2bcc2f1212d2a1"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.693987 23087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:34.694679 23111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.694890 23087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:34.694985 23087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "70befd223ade41fea4a7e017fcdd1dcd"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "7305abc8794f769cd6922440bdcffdfa"
server_key_iv: "ffe52034aa95e2119c2bcc2f1212d2a1"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.695067 23087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:34.703689 23087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:34.703956 23087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:34.704107 23087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:34.704324 23087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:34.704638 23087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:34.704691 23087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.704731 23087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:34.704757 23087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.710301 23087 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:33835
I20260812 03:48:34.710338 23279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:33835 every 8 connection(s)
I20260812 03:48:34.710656 23087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:34.714119 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 23087
I20260812 03:48:34.714241 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:34.714489 7698 external_mini_cluster.cc:1428] Setting key 592f81e253655cb6fcb80e6a97e5d7d0
I20260812 03:48:34.715939 22694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.711434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33639 (local address 127.7.132.190:41841)
0812 03:48:34.711530 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:34.711532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:34.711783 (+ 251us) server_negotiation.cc:407] Connection header received
0812 03:48:34.712238 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:34.712239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:34.712251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:34.712310 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:34.712921 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.713383 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.714387 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.714698 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.714805 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.715675 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:34.715676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:34.715677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:34.715681 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:34.715695 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:34.715699 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:34.715751 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:34.715837 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 03:48:34.715868 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:48:34.716125 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.711372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41841 (local address 127.7.132.131:33639)
0812 03:48:34.711644 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:34.711683 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:34.712154 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:34.712356 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:34.712359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:34.712368 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:34.712852 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:34.712856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.713458 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.713458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:34.714294 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:34.714297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:34.714827 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:34.714829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:34.714980 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:34.715529 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:34.715529 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:34.715557 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:34.715578 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:34.715768 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:34.715772 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:34.715775 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:34.715831 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:48:34.715921 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":175,"threads_started":1}
I20260812 03:48:34.716631 23280 heartbeater.cc:347] Connected to a master server at 127.7.132.190:41841
I20260812 03:48:34.716734 23280 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:34.716928 23280 heartbeater.cc:511] Master 127.7.132.190:41841 requested a full tablet report, sending...
I20260812 03:48:34.717316 22624 ts_manager.cc:194] Registered new tserver with Master: 70befd223ade41fea4a7e017fcdd1dcd (127.7.132.131:33835)
I20260812 03:48:34.717724 22624 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.131:33639
I20260812 03:48:34.726347 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:48:34.728149 22694 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:34.727657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.7.132.190:41841)
0812 03:48:34.727733 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:34.727734 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:34.727840 (+ 106us) server_negotiation.cc:407] Connection header received
0812 03:48:34.727900 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:34.727901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:34.727913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:34.727963 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:34.728088 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55360: BlockingRecv error: recv got EOF from 127.0.0.1:55360 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:48:34.728211 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 22704
I20260812 03:48:34.747089 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 22904
I20260812 03:48:34.759332 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 23087
I20260812 03:48:34.768127 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 22569
2026-08-12T03:48:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (628 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T03:48:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:34Z Disabled control of system clock
I20260812 03:48:34.802670 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34487
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:34487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:48:34.881467 23298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:34.881652 23298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:34.881673 23298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:34.882709 23298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:34.882762 23298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:34.882774 23298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:34.882786 23298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:34.882798 23298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:34.884107 23298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:34487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:34.884312 23298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:34.884553 23298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:34.889240 23309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.889236 23307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:34.889241 23305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:34.889312 23298 server_base.cc:1034] running on GCE node
I20260812 03:48:34.889747 23298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:34.890044 23298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:34.891213 23298 hybrid_clock.cc:648] HybridClock initialized: now 1786506514891208 us; error 38 us; skew 500 ppm
I20260812 03:48:34.892416 23298 webserver.cc:460] Webserver started at http://127.7.132.190:34175/ using document root <none> and password file <none>
I20260812 03:48:34.892674 23298 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:34.892746 23298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:34.892912 23298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:34.893981 23298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "84b324162ad542d8a0392853a1c98337"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "a0b7f747a105c3b4e54150ea94b42dfd"
server_key_iv: "397a0645827aee607f04ae2795f80647"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.894347 23298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "84b324162ad542d8a0392853a1c98337"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "a0b7f747a105c3b4e54150ea94b42dfd"
server_key_iv: "397a0645827aee607f04ae2795f80647"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.895727 23298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:34.896431 23318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:34.896637 23298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:34.896734 23298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "84b324162ad542d8a0392853a1c98337"
format_stamp: "Formatted at 2026-08-12 03:48:34 on dist-test-slave-39l8"
server_key: "a0b7f747a105c3b4e54150ea94b42dfd"
server_key_iv: "397a0645827aee607f04ae2795f80647"
server_key_version: "encryptionkey@0"
I20260812 03:48:34.896814 23298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:34.900100 23298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:34.900357 23298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:34.900496 23298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:34.904312 23298 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:34487
I20260812 03:48:34.904364 23388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:34487 every 8 connection(s)
I20260812 03:48:34.904700 23298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:34.905407 23391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:34.906869 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 23298
I20260812 03:48:34.906980 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:34.907181 7698 external_mini_cluster.cc:1428] Setting key 8a9ddd6d8b2fe99ecf6b7ac0be9e07d7
I20260812 03:48:34.909237 23391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337: Bootstrap starting.
I20260812 03:48:34.909567 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:34.907700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.7.132.190:34487)
0812 03:48:34.907889 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:34.907895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:34.907918 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:48:34.908312 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:34.908318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:34.908354 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:34.908458 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:34.909168 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:34.909172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:34.909172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:34.909176 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:34.909212 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:34.909221 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:34.909281 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:34.909372 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 03:48:34.909433 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260812 03:48:34.910035 23391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:34.910375 23391 log.cc:824] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:34.911176 23391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337: No bootstrap required, opened a new log
I20260812 03:48:34.912540 23391 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b324162ad542d8a0392853a1c98337" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34487 } }
I20260812 03:48:34.912674 23391 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:34.912724 23391 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b324162ad542d8a0392853a1c98337, State: Initialized, Role: FOLLOWER
I20260812 03:48:34.912847 23391 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b324162ad542d8a0392853a1c98337" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34487 } }
I20260812 03:48:34.912930 23391 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:34.912976 23391 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:34.913023 23391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:34.913587 23391 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b324162ad542d8a0392853a1c98337" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34487 } }
I20260812 03:48:34.913705 23391 leader_election.cc:302] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84b324162ad542d8a0392853a1c98337; no voters:
I20260812 03:48:34.913887 23391 leader_election.cc:288] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:34.914184 23391 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:34.914182 23403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:34.914352 23403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [term 1 LEADER]: Becoming Leader. State: Replica: 84b324162ad542d8a0392853a1c98337, State: Running, Role: LEADER
I20260812 03:48:34.914463 23403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b324162ad542d8a0392853a1c98337" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34487 } }
I20260812 03:48:34.914820 23405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84b324162ad542d8a0392853a1c98337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b324162ad542d8a0392853a1c98337" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34487 } } }
I20260812 03:48:34.914921 23405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:34.915047 23407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84b324162ad542d8a0392853a1c98337. Latest consensus state: current_term: 1 leader_uuid: "84b324162ad542d8a0392853a1c98337" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84b324162ad542d8a0392853a1c98337" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34487 } } }
I20260812 03:48:34.915118 23407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337 [sys.catalog]: This master's current role is: LEADER
W20260812 03:48:34.916136 23428 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:48:34.916183 23428 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:48:34.916246 23418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:34.916378 23418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:34.917680 23418 catalog_manager.cc:1347] Generated new cluster ID: 84b46c5449b140debb4fef0ededf598a
I20260812 03:48:34.917732 23418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:48:34.928812 23418 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:34.928980 23418 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:34.933175 23418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84b324162ad542d8a0392853a1c98337: Generated new TSK 0
I20260812 03:48:34.933352 23418 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:48:34.939814 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34487
--builtin_ntp_servers=127.7.132.148:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:48:35.020445 23431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:35.020617 23431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:35.020644 23431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:35.021685 23431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:35.021723 23431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:35.021744 23431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:35.023061 23431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:35.023304 23431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:35.023598 23431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:35.028434 23440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:35.028640 23439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:35.028579 23431 server_base.cc:1034] running on GCE node
W20260812 03:48:35.028472 23443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:35.028839 23431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:35.029078 23431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:35.030216 23431 hybrid_clock.cc:648] HybridClock initialized: now 1786506515030195 us; error 36 us; skew 500 ppm
I20260812 03:48:35.031468 23431 webserver.cc:460] Webserver started at http://127.7.132.129:45101/ using document root <none> and password file <none>
I20260812 03:48:35.031720 23431 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:35.031766 23431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:35.031912 23431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:35.032912 23431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "6afa169f466d4a15995a3a4ac80520be"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "22bc0ae10686b4be28394c4cf9c43efb"
server_key_iv: "ed593c5ca920cd78ee684bcbf1b3a751"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.033262 23431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "6afa169f466d4a15995a3a4ac80520be"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "22bc0ae10686b4be28394c4cf9c43efb"
server_key_iv: "ed593c5ca920cd78ee684bcbf1b3a751"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.034644 23431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:35.035445 23453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.035647 23431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:35.035737 23431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "6afa169f466d4a15995a3a4ac80520be"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "22bc0ae10686b4be28394c4cf9c43efb"
server_key_iv: "ed593c5ca920cd78ee684bcbf1b3a751"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.035828 23431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:35.038899 23431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:35.039168 23431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:35.039307 23431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:35.039520 23431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:35.039812 23431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:35.039865 23431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.039901 23431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:35.039927 23431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.045542 23431 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:33371
I20260812 03:48:35.045588 23619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:33371 every 8 connection(s)
I20260812 03:48:35.045871 23431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:48:35.048481 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:35.046513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:54445 (local address 127.7.132.190:34487)
0812 03:48:35.046596 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:35.046598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:35.046868 (+ 270us) server_negotiation.cc:407] Connection header received
0812 03:48:35.047273 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:35.047275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:35.047280 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:35.047328 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:35.048286 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:35.048288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:35.048288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:35.048292 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:35.048306 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:35.048309 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:35.048352 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:35.048424 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:48:35.048435 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:48:35.048664 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:35.046484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34487 (local address 127.7.132.129:54445)
0812 03:48:35.046691 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:35.046789 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:35.047193 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:35.047373 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:35.047376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:35.047383 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:35.048143 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:35.048144 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:35.048166 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:35.048191 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:35.048364 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:35.048366 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:35.048368 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:48:35.048416 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 03:48:35.048459 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260812 03:48:35.049175 23620 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34487
I20260812 03:48:35.049263 23620 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:35.049437 23620 heartbeater.cc:511] Master 127.7.132.190:34487 requested a full tablet report, sending...
I20260812 03:48:35.049973 23339 ts_manager.cc:194] Registered new tserver with Master: 6afa169f466d4a15995a3a4ac80520be (127.7.132.129:33371)
I20260812 03:48:35.055361 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 23431
I20260812 03:48:35.055454 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:35.055640 7698 external_mini_cluster.cc:1428] Setting key 089620cb2cac9e9402136666d3ee14d1
I20260812 03:48:35.056509 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34487
--builtin_ntp_servers=127.7.132.148:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:48:35.135747 23627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:35.135959 23627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:35.135993 23627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:35.137097 23627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:35.137166 23627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:35.137208 23627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:35.138599 23627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:35.138865 23627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:35.139189 23627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:35.143847 23639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:35.143857 23641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:35.143877 23637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:35.144009 23627 server_base.cc:1034] running on GCE node
I20260812 03:48:35.144217 23627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:35.144452 23627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:35.145591 23627 hybrid_clock.cc:648] HybridClock initialized: now 1786506515145581 us; error 29 us; skew 500 ppm
I20260812 03:48:35.146857 23627 webserver.cc:460] Webserver started at http://127.7.132.130:44971/ using document root <none> and password file <none>
I20260812 03:48:35.147090 23627 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:35.147133 23627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:35.147269 23627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:35.148258 23627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "1dd4442fa9d0466794af363fc20a986e"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "1e8af88883818df4243b755093e76ef7"
server_key_iv: "aac096a3e70d6c7ffc52a8321aa9adc9"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.148602 23627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "1dd4442fa9d0466794af363fc20a986e"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "1e8af88883818df4243b755093e76ef7"
server_key_iv: "aac096a3e70d6c7ffc52a8321aa9adc9"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.149854 23627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:48:35.150540 23648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.150745 23627 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:48:35.150851 23627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "1dd4442fa9d0466794af363fc20a986e"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "1e8af88883818df4243b755093e76ef7"
server_key_iv: "aac096a3e70d6c7ffc52a8321aa9adc9"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.150941 23627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:35.154013 23627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:35.154270 23627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:35.154413 23627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:35.154615 23627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:35.154958 23627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:35.155009 23627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.155045 23627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:35.155071 23627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.160514 23627 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:44573
I20260812 03:48:35.160604 23825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:44573 every 8 connection(s)
I20260812 03:48:35.160894 23627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:48:35.161674 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 23627
I20260812 03:48:35.161778 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:48:35.161978 7698 external_mini_cluster.cc:1428] Setting key 34a0d2a2a9aba7de0e115f7ab9cd44dd
I20260812 03:48:35.162967 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34487
--builtin_ntp_servers=127.7.132.148:36291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 03:48:35.163897 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:35.161607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:45209 (local address 127.7.132.190:34487)
0812 03:48:35.161670 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:35.161672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:35.161870 (+ 198us) server_negotiation.cc:407] Connection header received
0812 03:48:35.162413 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:35.162414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:35.162420 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:35.162466 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:35.163628 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:35.163629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:35.163630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:35.163635 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:35.163649 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:35.163653 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:35.163735 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:35.163809 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:48:35.163824 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:48:35.164111 23828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:35.161595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34487 (local address 127.7.132.130:45209)
0812 03:48:35.161764 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:35.161800 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:35.162325 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:35.162528 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:35.162532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:35.162541 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:35.163488 (+ 947us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:35.163488 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:35.163514 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:35.163539 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:35.163729 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:35.163733 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:35.163736 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:35.163813 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:48:35.163870 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260812 03:48:35.164708 23826 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34487
I20260812 03:48:35.164803 23826 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:35.164973 23826 heartbeater.cc:511] Master 127.7.132.190:34487 requested a full tablet report, sending...
I20260812 03:48:35.165333 23339 ts_manager.cc:194] Registered new tserver with Master: 1dd4442fa9d0466794af363fc20a986e (127.7.132.130:44573)
W20260812 03:48:35.243665 23829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:35.243873 23829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:35.243896 23829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:35.245048 23829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:35.245090 23829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:35.245146 23829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:35.246505 23829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:34487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:35.246713 23829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:35.247023 23829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:35.251858 23837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:35.251863 23838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:35.251895 23843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:35.252074 23829 server_base.cc:1034] running on GCE node
I20260812 03:48:35.252367 23829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:35.252588 23829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:35.253724 23829 hybrid_clock.cc:648] HybridClock initialized: now 1786506515253700 us; error 40 us; skew 500 ppm
I20260812 03:48:35.255218 23829 webserver.cc:460] Webserver started at http://127.7.132.131:32893/ using document root <none> and password file <none>
I20260812 03:48:35.255457 23829 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:35.255503 23829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:35.255656 23829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:35.256713 23829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "bb96f9d58aea4ff7b888e7f798cb9c8e"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "b9f15cb8f6dcc3b38fee74d0a47beeea"
server_key_iv: "044fada258ef23876b941ce9bdccd77f"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.257083 23829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "bb96f9d58aea4ff7b888e7f798cb9c8e"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "b9f15cb8f6dcc3b38fee74d0a47beeea"
server_key_iv: "044fada258ef23876b941ce9bdccd77f"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.258338 23829 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:48:35.259083 23854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.259271 23829 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.259361 23829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "bb96f9d58aea4ff7b888e7f798cb9c8e"
format_stamp: "Formatted at 2026-08-12 03:48:35 on dist-test-slave-39l8"
server_key: "b9f15cb8f6dcc3b38fee74d0a47beeea"
server_key_iv: "044fada258ef23876b941ce9bdccd77f"
server_key_version: "encryptionkey@0"
I20260812 03:48:35.259433 23829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:35.262686 23829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:35.262990 23829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:35.263118 23829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:35.263335 23829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:35.263648 23829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:35.263679 23829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.263751 23829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:35.263780 23829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:35.269428 23829 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:33871
I20260812 03:48:35.269464 24019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:33871 every 8 connection(s)
I20260812 03:48:35.269788 23829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:35.272830 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:35.270525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41165 (local address 127.7.132.190:34487)
0812 03:48:35.270586 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:35.270588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:35.270859 (+ 271us) server_negotiation.cc:407] Connection header received
0812 03:48:35.271350 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:35.271351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:35.271358 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:35.271408 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:35.272512 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:35.272514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:35.272514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:48:35.272520 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:35.272535 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:48:35.272539 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:35.272611 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:35.272710 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 03:48:35.272726 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:48:35.273058 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:35.270518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:34487 (local address 127.7.132.131:41165)
0812 03:48:35.270697 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:35.270773 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:35.271240 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:35.271473 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:35.271481 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:35.271489 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:35.272364 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:48:35.272365 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:35.272392 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:48:35.272418 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:35.272612 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:35.272617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:35.272620 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:35.272706 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:48:35.272825 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260812 03:48:35.273787 24020 heartbeater.cc:347] Connected to a master server at 127.7.132.190:34487
I20260812 03:48:35.273897 24020 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:35.274049 24020 heartbeater.cc:511] Master 127.7.132.190:34487 requested a full tablet report, sending...
I20260812 03:48:35.274432 23339 ts_manager.cc:194] Registered new tserver with Master: bb96f9d58aea4ff7b888e7f798cb9c8e (127.7.132.131:33871)
I20260812 03:48:35.278918 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 23829
I20260812 03:48:35.279001 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:35.279213 7698 external_mini_cluster.cc:1428] Setting key 93db7692dcf6e999a5c45efa8e51c4c0
I20260812 03:48:35.280187 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:48:35.282207 23398 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:48:35.281634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39894 (local address 127.7.132.190:34487)
0812 03:48:35.281711 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:35.281713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:35.281847 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:48:35.281927 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:35.281928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:35.281935 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:35.281991 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:35.282118 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39894: BlockingRecv error: recv got EOF from 127.0.0.1:39894 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:48:35.282292 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 23431
I20260812 03:48:35.299602 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 23627
I20260812 03:48:35.310463 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 23829
I20260812 03:48:35.332315 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 23298
2026-08-12T03:48:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (563 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:48:35 dist-test-slave-39l8 krb5kdc[24036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:48:35 dist-test-slave-39l8 krb5kdc[24036](info): set up 2 sockets
Aug 12 03:48:35 dist-test-slave-39l8 krb5kdc[24036](info): commencing operation
krb5kdc: starting...
W20260812 03:48:36.537604 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:48:36 dist-test-slave-39l8 krb5kdc[24036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:48:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:48:36.655714 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42953
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:36.734751 24053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:36.734956 24053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:36.734978 24053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:36.736106 24053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:48:36.736145 24053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:36.736159 24053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:36.736172 24053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:48:36.736229 24053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:48:36.737593 24053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:36.737812 24053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:36.738021 24053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:36.742592 24059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:36.742600 24061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:36.742600 24058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:36.742830 24053 server_base.cc:1034] running on GCE node
I20260812 03:48:36.743119 24053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:36.743438 24053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:36.744578 24053 hybrid_clock.cc:648] HybridClock initialized: now 1786506516744562 us; error 34 us; skew 500 ppm
Aug 12 03:48:36 dist-test-slave-39l8 krb5kdc[24036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506516, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:36.751920 24053 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:48:36.752357 24053 webserver.cc:460] Webserver started at http://127.7.132.190:37827/ using document root <none> and password file <none>
I20260812 03:48:36.752604 24053 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:36.752676 24053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:36.752830 24053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:36.753857 24053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "96734a4fddbe4e268106c0886b265c76"
format_stamp: "Formatted at 2026-08-12 03:48:36 on dist-test-slave-39l8"
server_key: "f3072eb45290426fd4569c209a360949"
server_key_iv: "168c6dffdbc2e91702b451b7211f9952"
server_key_version: "encryptionkey@0"
I20260812 03:48:36.754226 24053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "96734a4fddbe4e268106c0886b265c76"
format_stamp: "Formatted at 2026-08-12 03:48:36 on dist-test-slave-39l8"
server_key: "f3072eb45290426fd4569c209a360949"
server_key_iv: "168c6dffdbc2e91702b451b7211f9952"
server_key_version: "encryptionkey@0"
I20260812 03:48:36.755683 24053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:48:36.756351 24070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:36.756513 24053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:36.756610 24053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "96734a4fddbe4e268106c0886b265c76"
format_stamp: "Formatted at 2026-08-12 03:48:36 on dist-test-slave-39l8"
server_key: "f3072eb45290426fd4569c209a360949"
server_key_iv: "168c6dffdbc2e91702b451b7211f9952"
server_key_version: "encryptionkey@0"
I20260812 03:48:36.756690 24053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:36.777606 24053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:36.779242 24053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:36.779399 24053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:36.783252 24053 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42953
I20260812 03:48:36.783283 24148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42953 every 8 connection(s)
I20260812 03:48:36.783587 24053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:48:36.784596 24150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:36.786783 24150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76: Bootstrap starting.
I20260812 03:48:36.787319 24150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:36.787545 24150 log.cc:824] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:36.788151 24150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76: No bootstrap required, opened a new log
I20260812 03:48:36.789592 24150 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96734a4fddbe4e268106c0886b265c76" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42953 } }
I20260812 03:48:36.789717 24150 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:36.789748 24150 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96734a4fddbe4e268106c0886b265c76, State: Initialized, Role: FOLLOWER
I20260812 03:48:36.789881 24150 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96734a4fddbe4e268106c0886b265c76" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42953 } }
I20260812 03:48:36.789964 24150 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:48:36.789987 24150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:48:36.790014 24150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:36.790752 24150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96734a4fddbe4e268106c0886b265c76" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42953 } }
I20260812 03:48:36.790798 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 24053
I20260812 03:48:36.790866 24150 leader_election.cc:302] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96734a4fddbe4e268106c0886b265c76; no voters:
I20260812 03:48:36.790904 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:48:36.791025 24150 leader_election.cc:288] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:48:36.791049 24154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:36.791103 7698 external_mini_cluster.cc:1428] Setting key d92d049e78ba6845fe7cb60ab01c2363
I20260812 03:48:36.791157 24154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [term 1 LEADER]: Becoming Leader. State: Replica: 96734a4fddbe4e268106c0886b265c76, State: Running, Role: LEADER
I20260812 03:48:36.791226 24154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96734a4fddbe4e268106c0886b265c76" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42953 } }
I20260812 03:48:36.791265 24150 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:48:36.791538 24154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96734a4fddbe4e268106c0886b265c76. Latest consensus state: current_term: 1 leader_uuid: "96734a4fddbe4e268106c0886b265c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96734a4fddbe4e268106c0886b265c76" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42953 } } }
I20260812 03:48:36.791608 24154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:36.791549 24157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96734a4fddbe4e268106c0886b265c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96734a4fddbe4e268106c0886b265c76" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42953 } } }
I20260812 03:48:36.791653 24157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76 [sys.catalog]: This master's current role is: LEADER
I20260812 03:48:36.792132 24163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:48:36.792732 24163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:48:36.794338 24163 catalog_manager.cc:1347] Generated new cluster ID: 5e1aad2cd34f4f42b885fc1a8660cd7a
I20260812 03:48:36.794389 24163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:48:36 dist-test-slave-39l8 krb5kdc[24036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:36.803026 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:36.791664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.7.132.190:42953)
0812 03:48:36.791904 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:36.791913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:36.792174 (+ 261us) server_negotiation.cc:407] Connection header received
0812 03:48:36.792289 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:36.792297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:36.792369 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:36.792500 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:36.793454 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:36.794832 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:36.795841 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:36.796410 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:36.796506 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:36.799062 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:36.799066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:36.799066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:36.799077 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:36.801438 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:36.801814 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:36.801816 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:36.801817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:36.801925 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:36.802113 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:36.802114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:36.802114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:36.802406 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:36.802554 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:36.802650 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:48:36.802832 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":89,"threads_started":1}
I20260812 03:48:36.841934 24163 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:48:36.842685 24163 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:48:36.850881 24163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76: Generated new TSK 0
I20260812 03:48:36.851043 24163 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:48:36.883469 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42953
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:36.962597 24192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:36.962862 24192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:36.962891 24192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:36.964022 24192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:36.964061 24192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:36.964087 24192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:48:36.965582 24192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:36.965854 24192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:36.966118 24192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:36.970861 24199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:36.970846 24200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:36.970881 24192 server_base.cc:1034] running on GCE node
W20260812 03:48:36.970846 24202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:36.971235 24192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:36.971508 24192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:36.972692 24192 hybrid_clock.cc:648] HybridClock initialized: now 1786506516972671 us; error 35 us; skew 500 ppm
Aug 12 03:48:36 dist-test-slave-39l8 krb5kdc[24036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506516, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:36.975317 24192 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:48:36.975672 24192 webserver.cc:460] Webserver started at http://127.7.132.129:34925/ using document root <none> and password file <none>
I20260812 03:48:36.975940 24192 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:36.976009 24192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:36.976177 24192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:36.977180 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "1a5ab9e31c7140cc8f142b22e01812c2"
format_stamp: "Formatted at 2026-08-12 03:48:36 on dist-test-slave-39l8"
server_key: "896be88b7213a842ce9728805f98732c"
server_key_iv: "0e9e458429d39b8ed8e0f8fce5099e86"
server_key_version: "encryptionkey@0"
I20260812 03:48:36.977536 24192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "1a5ab9e31c7140cc8f142b22e01812c2"
format_stamp: "Formatted at 2026-08-12 03:48:36 on dist-test-slave-39l8"
server_key: "896be88b7213a842ce9728805f98732c"
server_key_iv: "0e9e458429d39b8ed8e0f8fce5099e86"
server_key_version: "encryptionkey@0"
I20260812 03:48:36.978796 24192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:48:36.979543 24209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:36.979727 24192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:36.979827 24192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "1a5ab9e31c7140cc8f142b22e01812c2"
format_stamp: "Formatted at 2026-08-12 03:48:36 on dist-test-slave-39l8"
server_key: "896be88b7213a842ce9728805f98732c"
server_key_iv: "0e9e458429d39b8ed8e0f8fce5099e86"
server_key_version: "encryptionkey@0"
I20260812 03:48:36.979910 24192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:36.987775 24192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:36.989454 24192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:36.989615 24192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:36.989818 24192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:36.990125 24192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:36.990178 24192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:36.990216 24192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:36.990248 24192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:36.996335 24192 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:46747
I20260812 03:48:36.996379 24372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:46747 every 8 connection(s)
I20260812 03:48:36.996634 24192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:48:36.997908 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 24192
I20260812 03:48:36.997999 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:48:36.998193 7698 external_mini_cluster.cc:1428] Setting key a341c2a158398268e4bd02aa75b25906
Aug 12 03:48:36 dist-test-slave-39l8 krb5kdc[24036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506516, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:37.006897 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:36.997248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:40811 (local address 127.7.132.190:42953)
0812 03:48:36.997291 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:36.997293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:36.997473 (+ 180us) server_negotiation.cc:407] Connection header received
0812 03:48:36.997971 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:36.997972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:36.997985 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:36.998032 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:36.999134 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:36.999586 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.000870 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.001067 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.001128 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.003657 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:37.003658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:37.003659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:37.003665 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:37.005485 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:37.005882 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:37.005883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:37.005884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:37.005952 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:37.006191 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:37.006192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:37.006192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:37.006350 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:37.006426 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.006669 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.006787 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:48:37.006925 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:36.997210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42953 (local address 127.7.132.129:40811)
0812 03:48:36.997365 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:36.997403 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:36.997854 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:36.998144 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:36.998147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:36.998839 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:36.999041 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:36.999046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:36.999716 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:36.999717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.000774 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.000777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.001166 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.001167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.001311 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.001812 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:37.001820 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:37.003559 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:37.005582 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:37.005589 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:37.005590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:37.005802 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:37.006025 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:37.006026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:37.006026 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:37.006118 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:37.006420 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:37.006424 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:37.006524 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 03:48:37.006600 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:48:37.006767 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260812 03:48:37.007539 24375 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42953
I20260812 03:48:37.007640 24375 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:37.007844 24375 heartbeater.cc:511] Master 127.7.132.190:42953 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
I20260812 03:48:37.008369 24098 ts_manager.cc:194] Registered new tserver with Master: 1a5ab9e31c7140cc8f142b22e01812c2 (127.7.132.129:46747)
I20260812 03:48:37.009001 24098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:40811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:48:37.023734 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42953
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:37.103596 24383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:37.103792 24383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:37.103816 24383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:37.104899 24383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:37.104933 24383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:37.104959 24383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:48:37.106293 24383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:37.106462 24383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:37.106626 24383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:37.111498 24390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:37.111495 24396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:37.111692 24394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:37.111689 24383 server_base.cc:1034] running on GCE node
I20260812 03:48:37.111936 24383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:37.112142 24383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:37.113283 24383 hybrid_clock.cc:648] HybridClock initialized: now 1786506517113269 us; error 31 us; skew 500 ppm
Aug 12 03:48:37 dist-test-slave-39l8 krb5kdc[24036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506517, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:37.115808 24383 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:48:37.116277 24383 webserver.cc:460] Webserver started at http://127.7.132.130:41169/ using document root <none> and password file <none>
I20260812 03:48:37.116508 24383 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:37.116573 24383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:37.116719 24383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:37.117710 24383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "471d618ef5ac4163a338b391c5dcd2bd"
format_stamp: "Formatted at 2026-08-12 03:48:37 on dist-test-slave-39l8"
server_key: "ed0673fbe44e8753fbae8ed53dc4c774"
server_key_iv: "c688e28067d6869543fc8f15a88e4620"
server_key_version: "encryptionkey@0"
I20260812 03:48:37.118090 24383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "471d618ef5ac4163a338b391c5dcd2bd"
format_stamp: "Formatted at 2026-08-12 03:48:37 on dist-test-slave-39l8"
server_key: "ed0673fbe44e8753fbae8ed53dc4c774"
server_key_iv: "c688e28067d6869543fc8f15a88e4620"
server_key_version: "encryptionkey@0"
I20260812 03:48:37.119472 24383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:48:37.120132 24404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:37.120306 24383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:37.120411 24383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "471d618ef5ac4163a338b391c5dcd2bd"
format_stamp: "Formatted at 2026-08-12 03:48:37 on dist-test-slave-39l8"
server_key: "ed0673fbe44e8753fbae8ed53dc4c774"
server_key_iv: "c688e28067d6869543fc8f15a88e4620"
server_key_version: "encryptionkey@0"
I20260812 03:48:37.120484 24383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:37.131166 24383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:37.132814 24383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:37.132963 24383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:37.133168 24383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:37.133463 24383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:37.133517 24383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:37.133555 24383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:37.133582 24383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:37.139321 24383 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:33103
I20260812 03:48:37.139369 24578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:33103 every 8 connection(s)
I20260812 03:48:37.139674 24383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:48:37 dist-test-slave-39l8 krb5kdc[24036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506517, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:37.148130 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.140389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:38565 (local address 127.7.132.190:42953)
0812 03:48:37.140455 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.140456 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.140634 (+ 178us) server_negotiation.cc:407] Connection header received
0812 03:48:37.141075 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.141076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.141087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.141139 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:37.142065 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.142546 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.143557 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.143735 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.143787 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.145857 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:37.145858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:37.145859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:37.145865 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:37.147013 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:37.147380 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:37.147380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:37.147381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:37.147426 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:37.147615 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:37.147616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:37.147616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:37.147723 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:37.147782 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.148010 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.148070 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:48:37.148376 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.140347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42953 (local address 127.7.132.130:38565)
0812 03:48:37.140516 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:37.140553 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:37.141000 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:37.141210 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:37.141213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:37.141806 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:37.141989 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.141993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.142622 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.142623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.143482 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.143484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.143808 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.143809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.143924 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.144436 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:37.144445 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:37.145778 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:37.147078 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:37.147084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:37.147086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:37.147304 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:37.147479 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:37.147479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:37.147480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:37.147566 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:37.147786 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:37.147790 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:37.147907 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 03:48:37.147986 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:48:37.148095 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260812 03:48:37.148363 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 24383
I20260812 03:48:37.148471 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:48:37.148687 7698 external_mini_cluster.cc:1428] Setting key c72c59d1ce64ad79d184a4ff17eeed5e
I20260812 03:48:37.148873 24579 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42953
I20260812 03:48:37.148979 24579 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:37.149160 24579 heartbeater.cc:511] Master 127.7.132.190:42953 requested a full tablet report, sending...
I20260812 03:48:37.149569 24098 ts_manager.cc:194] Registered new tserver with Master: 471d618ef5ac4163a338b391c5dcd2bd (127.7.132.130:33103)
I20260812 03:48:37.149955 24098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:38565
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:48:37.173125 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42953
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:43143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:48:37.252265 24586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:48:37.252460 24586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:48:37.252483 24586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:48:37.253639 24586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:48:37.253681 24586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:48:37.253736 24586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:48:37.255250 24586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:43143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:48:37.255474 24586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:48:37.255698 24586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:48:37.260394 24596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:37.260382 24593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:48:37.260408 24591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:48:37.260818 24586 server_base.cc:1034] running on GCE node
I20260812 03:48:37.261037 24586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:48:37.261310 24586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:48:37.262476 24586 hybrid_clock.cc:648] HybridClock initialized: now 1786506517262470 us; error 37 us; skew 500 ppm
Aug 12 03:48:37 dist-test-slave-39l8 krb5kdc[24036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506517, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:48:37.265008 24586 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:48:37.265475 24586 webserver.cc:460] Webserver started at http://127.7.132.131:38251/ using document root <none> and password file <none>
I20260812 03:48:37.265686 24586 fs_manager.cc:359] Metadata directory not provided
I20260812 03:48:37.265749 24586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:48:37.265889 24586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:48:37.267040 24586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "2a7bb124fba544d39736110c14ee050c"
format_stamp: "Formatted at 2026-08-12 03:48:37 on dist-test-slave-39l8"
server_key: "c326543ee323a6b8b457c65d5de2cac0"
server_key_iv: "a231dfacf9a40c0277b8c7c481e61d9b"
server_key_version: "encryptionkey@0"
I20260812 03:48:37.267405 24586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "2a7bb124fba544d39736110c14ee050c"
format_stamp: "Formatted at 2026-08-12 03:48:37 on dist-test-slave-39l8"
server_key: "c326543ee323a6b8b457c65d5de2cac0"
server_key_iv: "a231dfacf9a40c0277b8c7c481e61d9b"
server_key_version: "encryptionkey@0"
I20260812 03:48:37.268679 24586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:48:37.269331 24606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:37.269493 24586 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:48:37.269586 24586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "2a7bb124fba544d39736110c14ee050c"
format_stamp: "Formatted at 2026-08-12 03:48:37 on dist-test-slave-39l8"
server_key: "c326543ee323a6b8b457c65d5de2cac0"
server_key_iv: "a231dfacf9a40c0277b8c7c481e61d9b"
server_key_version: "encryptionkey@0"
I20260812 03:48:37.269661 24586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:48:37.287165 24586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:48:37.289289 24586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:48:37.289445 24586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:48:37.289660 24586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:48:37.289968 24586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:48:37.290019 24586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:37.290058 24586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:48:37.290086 24586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:48:37.296270 24586 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:38363
I20260812 03:48:37.296330 24764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:38363 every 8 connection(s)
I20260812 03:48:37.296613 24586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:48:37.298125 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 24586
I20260812 03:48:37.298252 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:48:37.298440 7698 external_mini_cluster.cc:1428] Setting key e90c7e14c9098c929e7dec7777c8e0ea
Aug 12 03:48:37 dist-test-slave-39l8 krb5kdc[24036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506517, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:48:37.305083 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.297254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:40477 (local address 127.7.132.190:42953)
0812 03:48:37.297324 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.297326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.297459 (+ 133us) server_negotiation.cc:407] Connection header received
0812 03:48:37.297880 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.297881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.297891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.297938 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:37.298938 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.299519 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.300411 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.300596 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.300663 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.302642 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:37.302643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:37.302644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:37.302651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:37.303982 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:37.304348 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:37.304349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:37.304349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:37.304397 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:37.304604 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:37.304605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:37.304605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:37.304695 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:37.304754 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.304966 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.305022 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:48:37.305233 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.297210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42953 (local address 127.7.132.131:40477)
0812 03:48:37.297356 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:37.297387 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:37.297807 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:37.297983 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:37.297986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:37.298587 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:48:37.298866 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.298873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.299604 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.299605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.300328 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.300330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.300669 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.300670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.300799 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.301253 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:48:37.301266 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:48:37.302561 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:48:37.304053 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:37.304059 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:37.304060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:37.304283 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:37.304455 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:48:37.304456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:48:37.304456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:48:37.304550 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:48:37.304753 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:48:37.304757 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:48:37.304859 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 03:48:37.304936 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:48:37.305047 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":94,"threads_started":1}
I20260812 03:48:37.305704 24765 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42953
I20260812 03:48:37.305810 24765 heartbeater.cc:464] Registering TS with master...
I20260812 03:48:37.305974 24765 heartbeater.cc:511] Master 127.7.132.190:42953 requested a full tablet report, sending...
I20260812 03:48:37.306356 24098 ts_manager.cc:194] Registered new tserver with Master: 2a7bb124fba544d39736110c14ee050c (127.7.132.131:38363)
I20260812 03:48:37.306756 24098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:40477
I20260812 03:48:37.310026 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:48:37.316409 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.311200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.7.132.190:42953)
0812 03:48:37.311256 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.311257 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.311398 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:48:37.311432 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.311433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.311440 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.311484 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:48:37.311948 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.312384 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.313031 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.313188 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.313233 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.314301 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:48:37.314302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:48:37.314302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:48:37.314307 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:48:37.315460 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:37.315767 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:37.315768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:37.315768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:37.315812 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:48:37.315991 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:48:37.315992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:48:37.315992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:48:37.316109 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:48:37.316179 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.316279 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.316329 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:48:37.317790 24098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:48:37.319379 24098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:48:37.326365 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.322034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46747 (local address 127.0.0.1:58694)
0812 03:48:37.322164 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:37.322203 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:37.322265 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:37.322659 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:37.322662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:37.322664 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:37.322752 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.322754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.324098 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.324099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.325339 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.325341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.326074 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.326075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.326158 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.326170 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:48:37.326253 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:48:37.326303 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":87,"threads_started":1}
I20260812 03:48:37.326403 24779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.322201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58694 (local address 127.7.132.129:46747)
0812 03:48:37.322398 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.322402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.322412 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:48:37.322468 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.322470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.322486 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.322563 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:37.322845 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.323983 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.325424 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.325993 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.326074 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.326128 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.326273 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.326314 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260812 03:48:37.326998 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.322636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:38363 (local address 127.0.0.1:34174)
0812 03:48:37.322780 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:37.322789 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:37.322861 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:37.323413 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:37.323414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:37.323416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:37.323462 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.323464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.324777 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.324778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.325709 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.325712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.326778 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.326780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.326853 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.326857 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:37.326888 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 03:48:37.326923 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":57,"threads_started":1}
I20260812 03:48:37.327332 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.323167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.7.132.130:33103)
0812 03:48:37.323349 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.323353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.323417 (+ 64us) server_negotiation.cc:407] Connection header received
0812 03:48:37.323503 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.323505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.323518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.323609 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:37.323714 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.324622 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.325737 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.326421 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.326854 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.326925 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.327000 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.327248 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260812 03:48:37.327237 24278 tablet_service.cc:1467] Processing CreateTablet for tablet ea276f4319384ddf81d1479790d192ce (DEFAULT_TABLE table=test-table [id=7b2f30bf15eb4d3daabe04d6c5a97f72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:37.327425 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.323032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.7.132.131:38363)
0812 03:48:37.323257 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.323261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.323270 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:37.323301 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.323304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.323317 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.323527 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:37.323577 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.324647 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.325796 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.326420 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.327218 (+ 798us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.327279 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.327304 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.327341 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":139,"threads_started":1}
I20260812 03:48:37.327427 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.322812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33103 (local address 127.0.0.1:54284)
0812 03:48:37.323365 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:37.323375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:37.323432 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:37.323616 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:37.323617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:37.323620 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:37.323661 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.323663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.324718 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.324719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.325645 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.325648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.326864 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.326865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.326924 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.326928 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:48:37.326988 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:48:37.327032 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":1,"thread_start_us":46,"threads_started":1}
I20260812 03:48:37.327607 24278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea276f4319384ddf81d1479790d192ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:37.327946 24669 tablet_service.cc:1467] Processing CreateTablet for tablet ea276f4319384ddf81d1479790d192ce (DEFAULT_TABLE table=test-table [id=7b2f30bf15eb4d3daabe04d6c5a97f72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:37.328250 24669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea276f4319384ddf81d1479790d192ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:37.330137 24789 tablet_bootstrap.cc:492] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2: Bootstrap starting.
I20260812 03:48:37.330605 24791 tablet_bootstrap.cc:492] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: Bootstrap starting.
I20260812 03:48:37.330678 24481 tablet_service.cc:1467] Processing CreateTablet for tablet ea276f4319384ddf81d1479790d192ce (DEFAULT_TABLE table=test-table [id=7b2f30bf15eb4d3daabe04d6c5a97f72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:48:37.330979 24789 tablet_bootstrap.cc:654] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:37.331058 24481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea276f4319384ddf81d1479790d192ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:48:37.331279 24791 tablet_bootstrap.cc:654] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:37.331532 24789 log.cc:824] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:37.331804 24791 log.cc:824] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:37.332268 24789 tablet_bootstrap.cc:492] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2: No bootstrap required, opened a new log
I20260812 03:48:37.332353 24789 ts_tablet_manager.cc:1397] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:37.333815 24795 tablet_bootstrap.cc:492] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Bootstrap starting.
I20260812 03:48:37.333781 24789 raft_consensus.cc:348] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.333983 24789 raft_consensus.cc:374] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:37.334026 24789 raft_consensus.cc:729] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a5ab9e31c7140cc8f142b22e01812c2, State: Initialized, Role: FOLLOWER
I20260812 03:48:37.334201 24789 consensus_queue.cc:260] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.334445 24795 tablet_bootstrap.cc:654] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Neither blocks nor log segments found. Creating new log.
I20260812 03:48:37.334535 24789 ts_tablet_manager.cc:1428] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:37.334539 24375 heartbeater.cc:503] Master 127.7.132.190:42953 was elected leader, sending a full tablet report...
I20260812 03:48:37.335052 24795 log.cc:824] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:48:37.335232 24791 tablet_bootstrap.cc:492] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: No bootstrap required, opened a new log
I20260812 03:48:37.335306 24791 ts_tablet_manager.cc:1397] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:48:37.335999 24795 tablet_bootstrap.cc:492] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: No bootstrap required, opened a new log
I20260812 03:48:37.336072 24795 ts_tablet_manager.cc:1397] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:48:37.337239 24791 raft_consensus.cc:348] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.337401 24791 raft_consensus.cc:374] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:37.337437 24791 raft_consensus.cc:729] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a7bb124fba544d39736110c14ee050c, State: Initialized, Role: FOLLOWER
I20260812 03:48:37.337539 24791 consensus_queue.cc:260] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.337817 24791 ts_tablet_manager.cc:1428] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:48:37.337815 24765 heartbeater.cc:503] Master 127.7.132.190:42953 was elected leader, sending a full tablet report...
I20260812 03:48:37.338035 24795 raft_consensus.cc:348] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.338164 24795 raft_consensus.cc:374] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:48:37.338208 24795 raft_consensus.cc:729] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471d618ef5ac4163a338b391c5dcd2bd, State: Initialized, Role: FOLLOWER
I20260812 03:48:37.338322 24795 consensus_queue.cc:260] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.338614 24579 heartbeater.cc:503] Master 127.7.132.190:42953 was elected leader, sending a full tablet report...
I20260812 03:48:37.338598 24795 ts_tablet_manager.cc:1428] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260812 03:48:37.390350 24580 tablet.cc:2369] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:48:37.472700 24797 raft_consensus.cc:482] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:48:37.472819 24797 raft_consensus.cc:504] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.473196 24797 leader_election.cc:288] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a7bb124fba544d39736110c14ee050c (127.7.132.131:38363), 471d618ef5ac4163a338b391c5dcd2bd (127.7.132.130:33103)
I20260812 03:48:37.476002 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.473418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:53227 (local address 127.7.132.130:33103)
0812 03:48:37.473469 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.473471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.473480 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:48:37.473534 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.473535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.473544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.473597 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:37.473826 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.474306 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.475267 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.475636 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.475730 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.475740 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.475902 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.475941 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:48:37.476002 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.473375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33103 (local address 127.7.132.129:53227)
0812 03:48:37.473415 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:37.473425 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:37.473467 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:37.473617 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:37.473618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:37.473620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:37.473713 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.473715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.474396 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.474398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.475161 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.475164 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.475729 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.475731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.475813 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.475816 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:48:37.475893 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:48:37.475931 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 03:48:37.476246 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.473447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:48361 (local address 127.7.132.131:38363)
0812 03:48:37.473489 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.473492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.473788 (+ 296us) server_negotiation.cc:407] Connection header received
0812 03:48:37.473842 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.473843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.473853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.473941 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:48:37.474096 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.474557 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.475554 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.475921 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.476005 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.476013 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.476153 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.476193 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:48:37.476267 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.473375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:38363 (local address 127.7.132.129:48361)
0812 03:48:37.473701 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:37.473710 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:48:37.473778 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:48:37.473943 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:48:37.473944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:48:37.473946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:48:37.474025 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.474027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.474651 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.474652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.475453 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:48:37.475455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.476006 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.476007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:48:37.476081 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.476086 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:48:37.476146 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:48:37.476189 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":146,"thread_start_us":114,"threads_started":1}
I20260812 03:48:37.476545 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea276f4319384ddf81d1479790d192ce" candidate_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471d618ef5ac4163a338b391c5dcd2bd" is_pre_election: true
I20260812 03:48:37.476699 24699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea276f4319384ddf81d1479790d192ce" candidate_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7bb124fba544d39736110c14ee050c" is_pre_election: true
I20260812 03:48:37.476747 24507 raft_consensus.cc:2454] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a5ab9e31c7140cc8f142b22e01812c2 in term 0.
I20260812 03:48:37.476830 24699 raft_consensus.cc:2454] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a5ab9e31c7140cc8f142b22e01812c2 in term 0.
I20260812 03:48:37.477052 24210 leader_election.cc:302] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a5ab9e31c7140cc8f142b22e01812c2, 471d618ef5ac4163a338b391c5dcd2bd; no voters:
I20260812 03:48:37.477181 24797 raft_consensus.cc:2781] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:48:37.477252 24797 raft_consensus.cc:482] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:48:37.477280 24797 raft_consensus.cc:3037] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:37.478101 24797 raft_consensus.cc:504] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.478251 24797 leader_election.cc:288] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [CANDIDATE]: Term 1 election: Requested vote from peers 2a7bb124fba544d39736110c14ee050c (127.7.132.131:38363), 471d618ef5ac4163a338b391c5dcd2bd (127.7.132.130:33103)
I20260812 03:48:37.478410 24699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea276f4319384ddf81d1479790d192ce" candidate_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a7bb124fba544d39736110c14ee050c"
I20260812 03:48:37.478434 24507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea276f4319384ddf81d1479790d192ce" candidate_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "471d618ef5ac4163a338b391c5dcd2bd"
I20260812 03:48:37.478485 24699 raft_consensus.cc:3037] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:37.478540 24507 raft_consensus.cc:3037] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:48:37.479285 24699 raft_consensus.cc:2454] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a5ab9e31c7140cc8f142b22e01812c2 in term 1.
I20260812 03:48:37.479408 24507 raft_consensus.cc:2454] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a5ab9e31c7140cc8f142b22e01812c2 in term 1.
I20260812 03:48:37.479478 24211 leader_election.cc:302] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a5ab9e31c7140cc8f142b22e01812c2, 2a7bb124fba544d39736110c14ee050c; no voters:
I20260812 03:48:37.479619 24797 raft_consensus.cc:2781] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:48:37.479744 24797 raft_consensus.cc:686] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1a5ab9e31c7140cc8f142b22e01812c2, State: Running, Role: LEADER
I20260812 03:48:37.479844 24797 consensus_queue.cc:237] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } }
I20260812 03:48:37.480630 24098 catalog_manager.cc:5495] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a5ab9e31c7140cc8f142b22e01812c2 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a5ab9e31c7140cc8f142b22e01812c2" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 46747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 } health_report { overall_health: UNKNOWN } } }
W20260812 03:48:37.497329 24377 tablet.cc:2369] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:48:37.507025 24779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.504262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.7.132.129:46747)
0812 03:48:37.504322 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:48:37.504324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:48:37.504340 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:48:37.504432 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:48:37.504434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:48:37.504454 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:48:37.504535 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:48:37.504666 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.505398 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.506183 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:48:37.506385 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:48:37.506462 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:48:37.506641 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:48:37.506767 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:48:37.506854 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:48:37.506911 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 03:48:37.506930 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:48:37.508553 24507 raft_consensus.cc:1261] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 1 FOLLOWER]: Refusing update from remote peer 1a5ab9e31c7140cc8f142b22e01812c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:37.508570 24699 raft_consensus.cc:1261] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 1 FOLLOWER]: Refusing update from remote peer 1a5ab9e31c7140cc8f142b22e01812c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:48:37.508912 24806 consensus_queue.cc:1035] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "471d618ef5ac4163a338b391c5dcd2bd" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:37.509030 24797 consensus_queue.cc:1035] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a7bb124fba544d39736110c14ee050c" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:48:37.511883 24811 mvcc.cc:205] Tried to move back new op lower bound from 7317530695712768000 to 7317530695597875200. Current Snapshot: MvccSnapshot[applied={T|T < 7317530695712768000}]
I20260812 03:48:37.513628 24098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:37.513703 24098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:48:37.514390 24098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96734a4fddbe4e268106c0886b265c76: Sending DeleteTablet for 3 replicas of tablet ea276f4319384ddf81d1479790d192ce
I20260812 03:48:37.514750 24669 tablet_service.cc:1514] Processing DeleteTablet for tablet ea276f4319384ddf81d1479790d192ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:37 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:34174
I20260812 03:48:37.514760 24278 tablet_service.cc:1514] Processing DeleteTablet for tablet ea276f4319384ddf81d1479790d192ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:37 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:58694
I20260812 03:48:37.515166 24481 tablet_service.cc:1514] Processing DeleteTablet for tablet ea276f4319384ddf81d1479790d192ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:48:37 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54284
I20260812 03:48:37.515256 24826 tablet_replica.cc:330] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2: stopping tablet replica
I20260812 03:48:37.515363 24826 raft_consensus.cc:2229] T ea276f4319384ddf81d1479790d192ce P 1a5ab9e31c7140cc8f142b22e01812c2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:48:37.515385 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 24192
I20260812 03:48:37.534847 24827 tablet_replica.cc:330] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: stopping tablet replica
I20260812 03:48:37.534991 24827 raft_consensus.cc:2229] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:37.535141 24827 raft_consensus.cc:2258] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:37.535315 24829 tablet_replica.cc:330] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: stopping tablet replica
I20260812 03:48:37.535420 24829 raft_consensus.cc:2229] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:48:37.535526 24829 raft_consensus.cc:2258] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:48:37.535629 24827 ts_tablet_manager.cc:1905] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:37.535957 24829 ts_tablet_manager.cc:1905] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:48:37.537317 24827 ts_tablet_manager.cc:1918] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:37.537397 24827 log.cc:1196] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/ea276f4319384ddf81d1479790d192ce
I20260812 03:48:37.537613 24829 ts_tablet_manager.cc:1918] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:48:37.537693 24829 log.cc:1196] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/ea276f4319384ddf81d1479790d192ce
I20260812 03:48:37.537698 24827 ts_tablet_manager.cc:1939] T ea276f4319384ddf81d1479790d192ce P 2a7bb124fba544d39736110c14ee050c: Deleting consensus metadata
I20260812 03:48:37.537969 24829 ts_tablet_manager.cc:1939] T ea276f4319384ddf81d1479790d192ce P 471d618ef5ac4163a338b391c5dcd2bd: Deleting consensus metadata
I20260812 03:48:37.538240 24074 catalog_manager.cc:4841] TS 2a7bb124fba544d39736110c14ee050c (127.7.132.131:38363): tablet ea276f4319384ddf81d1479790d192ce (table test-table [id=7b2f30bf15eb4d3daabe04d6c5a97f72]) successfully deleted
I20260812 03:48:37.538333 24072 catalog_manager.cc:4841] TS 471d618ef5ac4163a338b391c5dcd2bd (127.7.132.130:33103): tablet ea276f4319384ddf81d1479790d192ce (table test-table [id=7b2f30bf15eb4d3daabe04d6c5a97f72]) successfully deleted
W20260812 03:48:37.538247 24075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:46747 (error 108)
I20260812 03:48:37.538524 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 24383
I20260812 03:48:37.543478 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:48:37.543305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:46747 (local address 127.0.0.1:58714)
0812 03:48:37.543382 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 03:48:37.543434 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 03:48:37.543617 24075 catalog_manager.cc:4584] TS 1a5ab9e31c7140cc8f142b22e01812c2 (127.7.132.129:46747): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea276f4319384ddf81d1479790d192ce: Network error: Client connection negotiation failed: client connection to 127.7.132.129:46747: connect: Connection refused (error 111)
I20260812 03:48:37.547179 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 24586
I20260812 03:48:37.555866 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 24053
2026-08-12T03:48:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2220 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:48:37 dist-test-slave-39l8 krb5kdc[24835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:48:37 dist-test-slave-39l8 krb5kdc[24835](info): set up 2 sockets
Aug 12 03:48:37 dist-test-slave-39l8 krb5kdc[24835](info): commencing operation
krb5kdc: starting...
W20260812 03:48:38.752204 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:48:38 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:48:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:48:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/rangeradmin_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/rangeradmin_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/rangerlookup_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/rangerlookup_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
I20260812 03:48:38.906881 7698 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskWjyozu/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/postgres -l logfile start
2026-08-12 03:48:41.156 UTC [24890] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:48:41.157 UTC [24890] LOG: listening on IPv4 address "127.7.132.148", port 46305
2026-08-12 03:48:41.159 UTC [24890] LOG: listening on Unix socket "/tmp/.s.PGSQL.46305"
2026-08-12 03:48:41.194 UTC [24893] LOG: database system was shut down at 2026-08-12 03:48:39 UTC
2026-08-12 03:48:41.199 UTC [24890] LOG: database system is ready to accept connections
I20260812 03:48:42.321058 7698 mini_postgres.cc:96] Postgres bound to 46305
2026-08-12 03:48:42.329 UTC [24906] FATAL: database "slave" does not exist
127.7.132.148:46305 - accepting connections
I20260812 03:48:42.330380 7698 mini_ranger.cc:161] Starting Ranger...
I20260812 03:48:42.347486 7698 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:48:42.737812 7698 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:48:42.737880 7698 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-admin
2026-08-12 03:48:43,058 [I] DB FLAVOR :POSTGRES
2026-08-12 03:48:43,058 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:48:43,058 [I] Checking connection..
2026-08-12 03:48:43,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:48:43,298 [I] Checking connection passed.
2026-08-12 03:48:43,298 [I] --------- Verifying version history table ---------
2026-08-12 03:48:43,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:48:43,557 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:48:43,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:48:43,803 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:48:43,803 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:48:43,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:48:44.036 UTC [24997] WARNING: there is no transaction in progress
2026-08-12 03:48:44,044 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:48:44,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:48:44,296 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:48:44,296 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:48:44,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:48:44,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:48:44,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-39l8.c.gcp-upstream.internal','N') ;"
2026-08-12 03:48:44,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:48:45,237 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:48:45,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:48:45,463 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:48:45,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:48:45,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:48:45,943 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:48:45,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:48:46.811 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.825 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.840 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.853 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.867 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.883 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.931 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.937 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.951 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.961 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.971 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.983 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.991 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:46.997 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:47.003 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:47.009 UTC [25149] WARNING: there is no transaction in progress
2026-08-12 03:48:47,552 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:48:47,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:48:47,769 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:48:47,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:48:48,016 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:48:48,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:48:48,272 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:48:48,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:48:48,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:48:48,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-39l8.c.gcp-upstream.internal';"
2026-08-12 03:48:48,998 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:48:48,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:48:49,242 [I] DB_PATCHES have already been applied
I20260812 03:48:49.248322 7698 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-admin:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskWjyozu/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:48:49.248386 7698 mini_ranger.cc:193] Using host: 127.7.132.148
I20260812 03:48:49.252094 7698 mini_ranger.cc:227] Ranger admin URL: http://127.7.132.148:34223
Aug 12, 2026 3:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:48:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:48:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:48:50.617910 7698 mini_ranger.cc:160] Time spent starting Ranger: real 8.288s user 0.005s sys 0.006s
Aug 12, 2026 3:49:09 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:49:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 3:49:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:49:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:49:10 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:49:11.035494 7698 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/rangerkms_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/rangerkms_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.132.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.132.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:49:11 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506551, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:49:11.114038 7698 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:49:11.123133 7698 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:49:11.815950 7698 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:49:11.816033 7698 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms
I20260812 03:49:11.816056 7698 mini_ranger_kms.cc:227] Using postgres at 127.7.132.148:46305
2026-08-12 03:49:12,313 [I] DB FLAVOR :POSTGRES
2026-08-12 03:49:12,314 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:49:12,314 [I] Checking connection
2026-08-12 03:49:12,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:49:12,553 [I] connection success
2026-08-12 03:49:12,553 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:49:12,553 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:49:12,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:49:12,801 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:49:12,801 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:49:12,801 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:49:12,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:46305/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:49:13,047 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:49:13.480494 7698 mini_ranger_kms.cc:310] Created kms service
I20260812 03:49:13.650988 7698 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:49:13.693068 7698 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:49:13 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506551, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:49:14.073836 7698 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:49:14.073951 7698 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms:/tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:49:14.073971 7698 mini_ranger_kms.cc:240] Using host: 127.7.132.148
I20260812 03:49:14.079080 7698 mini_ranger_kms.cc:276] Ranger KMS PID: 25419
I20260812 03:49:14.079198 7698 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.132.148:33265
Aug 12, 2026 3:49:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:49:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:49:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:49:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:49:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:49:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:49:16 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:49:16.782459 7698 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.668s user 0.005s sys 0.257s
I20260812 03:49:16.782562 7698 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:49:16.782649 7698 mini_ranger_kms.cc:401] 127.7.132.148:33265/kms/v1/keys
Aug 12 03:49:16 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Aug 12 03:49:17 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506556, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Aug 12 03:49:19 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:49:19.425666 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42717
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42717
--ranger_config_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:33265/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:49:19.504936 25502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:49:19.505113 25502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:49:19.505133 25502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:49:19.506209 25502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:49:19.506247 25502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:49:19.506260 25502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:49:19.506271 25502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:49:19.506282 25502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:49:19.507905 25502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.132.148:33265/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42717
--ranger_config_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:49:19.508127 25502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:49:19.508358 25502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:49:19.512809 25509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:49:19.512782 25510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:49:19.512796 25513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:49:19.513034 25502 server_base.cc:1034] running on GCE node
I20260812 03:49:19.513388 25502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:49:19.513597 25502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:49:19.514776 25502 hybrid_clock.cc:648] HybridClock initialized: now 1786506559514753 us; error 41 us; skew 500 ppm
Aug 12 03:49:19 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506559, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:49:19.536926 25502 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:49:19.537720 25502 webserver.cc:460] Webserver started at http://127.7.132.190:34299/ using document root <none> and password file <none>
I20260812 03:49:19.537961 25502 fs_manager.cc:359] Metadata directory not provided
I20260812 03:49:19.538023 25502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:49:19.538201 25502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:49:19 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506559, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:49:19.588634 25502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3"
format_stamp: "Formatted at 2026-08-12 03:49:19 on dist-test-slave-39l8"
server_key: "4b57254455eb0811c97842524b9c79c7"
server_key_iv: "7cf35925f4b1cf98b54e09767916c898"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:19.589097 25502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3"
format_stamp: "Formatted at 2026-08-12 03:49:19 on dist-test-slave-39l8"
server_key: "4b57254455eb0811c97842524b9c79c7"
server_key_iv: "7cf35925f4b1cf98b54e09767916c898"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:19.590976 25502 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260812 03:49:19.602144 25520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:19.602344 25502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:49:19.602452 25502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3"
format_stamp: "Formatted at 2026-08-12 03:49:19 on dist-test-slave-39l8"
server_key: "4b57254455eb0811c97842524b9c79c7"
server_key_iv: "7cf35925f4b1cf98b54e09767916c898"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:19.602559 25502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:49:19.617254 25502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:49:19.619165 25502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:49:19.619350 25502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:49:19.625759 25502 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42717
I20260812 03:49:19.626212 25502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:49:19.632745 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 25502
I20260812 03:49:19.632851 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:49:19.633042 7698 external_mini_cluster.cc:1428] Setting key 617d0f6e7fc1223be352687861b653ed
I20260812 03:49:19.634742 25598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:49:19.634984 25597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42717 every 8 connection(s)
I20260812 03:49:19.637831 25598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3: Bootstrap starting.
I20260812 03:49:19.638576 25598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3: Neither blocks nor log segments found. Creating new log.
I20260812 03:49:19.638973 25598 log.cc:824] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:49:19.640421 25598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3: No bootstrap required, opened a new log
I20260812 03:49:19.642849 25598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42717 } }
I20260812 03:49:19.643039 25598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:49:19.643078 25598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ee7ac41a1942fd9ad29fc55d04bbe3, State: Initialized, Role: FOLLOWER
I20260812 03:49:19.643187 25598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42717 } }
I20260812 03:49:19.643253 25598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:49:19.643280 25598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:49:19.643321 25598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:49:19.643954 25598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42717 } }
I20260812 03:49:19.644059 25598 leader_election.cc:302] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33ee7ac41a1942fd9ad29fc55d04bbe3; no voters:
I20260812 03:49:19.644229 25598 leader_election.cc:288] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:49:19.644397 25605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:49:19.644469 25598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:49:19.644522 25605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [term 1 LEADER]: Becoming Leader. State: Replica: 33ee7ac41a1942fd9ad29fc55d04bbe3, State: Running, Role: LEADER
I20260812 03:49:19.644616 25605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42717 } }
Aug 12 03:49:19 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:49:19.644933 25605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42717 } } }
I20260812 03:49:19.644991 25605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:49:19.645118 25605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ee7ac41a1942fd9ad29fc55d04bbe3. Latest consensus state: current_term: 1 leader_uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33ee7ac41a1942fd9ad29fc55d04bbe3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42717 } } }
I20260812 03:49:19.645162 25605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:49:19.645973 25598 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:49:19.650199 25616 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:49:19.651374 25616 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:49:19.652794 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:19.635256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33332 (local address 127.7.132.190:42717)
0812 03:49:19.639289 (+ 4033us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:19.639298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:19.639329 (+ 31us) server_negotiation.cc:407] Connection header received
0812 03:49:19.639814 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:19.639822 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:19.639880 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:19.640025 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:49:19.640632 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:19.641588 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:19.642280 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:19.642657 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:19.642798 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:19.645373 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:49:19.645377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:49:19.645377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:49:19.645391 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:49:19.649573 (+ 4182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:19.650075 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:19.650079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:19.650080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:19.650244 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:19.650500 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:19.650501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:19.650502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:19.651163 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:49:19.651462 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:19.651977 (+ 515us) server_negotiation.cc:299] Negotiation successful
0812 03:49:19.652361 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3969,"thread_start_us":79,"threads_started":1}
I20260812 03:49:19.656265 25616 catalog_manager.cc:1347] Generated new cluster ID: 34b08b64ec8e486d870390c8496e6557
I20260812 03:49:19.656322 25616 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:49:19.675237 25616 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:49:19.676043 25616 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:49:19.691283 25616 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3: Generated new TSK 0
I20260812 03:49:19.691550 25616 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:49:20.438650 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:33265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42717
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:49:20.556962 25665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:49:20.557190 25665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:49:20.557221 25665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:49:20.558831 25665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:49:20.558895 25665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:49:20.558944 25665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:49:20.561018 25665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.132.148:33265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:49:20.561233 25665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:49:20.561491 25665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:49:20.566370 25677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:49:20.566411 25674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:49:20.566411 25673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:49:20.567057 25665 server_base.cc:1034] running on GCE node
I20260812 03:49:20.567257 25665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:49:20.567514 25665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:49:20.568675 25665 hybrid_clock.cc:648] HybridClock initialized: now 1786506560568663 us; error 30 us; skew 500 ppm
Aug 12 03:49:20 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:49:20.571359 25665 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:49:20.571744 25665 webserver.cc:460] Webserver started at http://127.7.132.129:35679/ using document root <none> and password file <none>
I20260812 03:49:20.571971 25665 fs_manager.cc:359] Metadata directory not provided
I20260812 03:49:20.572052 25665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:49:20.572228 25665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:49:20 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:49:20.583365 25665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "b4fec2785cbb41558b973dafd15c92f3"
format_stamp: "Formatted at 2026-08-12 03:49:20 on dist-test-slave-39l8"
server_key: "6f29a5a65ab9bb0819d452d4ca7f475e"
server_key_iv: "3688272652d47d7c0ac859a9673a35fd"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:20.583911 25665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "b4fec2785cbb41558b973dafd15c92f3"
format_stamp: "Formatted at 2026-08-12 03:49:20 on dist-test-slave-39l8"
server_key: "6f29a5a65ab9bb0819d452d4ca7f475e"
server_key_iv: "3688272652d47d7c0ac859a9673a35fd"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:20.585719 25665 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:49:20.599306 25688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:20.599499 25665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:49:20.599637 25665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "b4fec2785cbb41558b973dafd15c92f3"
format_stamp: "Formatted at 2026-08-12 03:49:20 on dist-test-slave-39l8"
server_key: "6f29a5a65ab9bb0819d452d4ca7f475e"
server_key_iv: "3688272652d47d7c0ac859a9673a35fd"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:20.599970 25665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:49:20.616920 25665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:49:20.619305 25665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:49:20.619455 25665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:49:20.619678 25665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:49:20.620069 25665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:49:20.620124 25665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:20.620155 25665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:49:20.620205 25665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:20.628650 25665 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:42995
I20260812 03:49:20.629122 25665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:49:20.630442 25875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:42995 every 8 connection(s)
I20260812 03:49:20.634864 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 25665
I20260812 03:49:20.634987 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:49:20.635181 7698 external_mini_cluster.cc:1428] Setting key 45038f8c7093912233fe78fee0556d74
Aug 12 03:49:20 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:49:20.639851 25880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:20.629995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42717 (local address 127.7.132.129:44061)
0812 03:49:20.630190 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:20.630235 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:49:20.631108 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:49:20.631393 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:49:20.631398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:49:20.631643 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:49:20.631912 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:20.631918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:20.632604 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:20.632605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:20.633870 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:20.633873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:20.634255 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:20.634256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:20.634393 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:20.635006 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:49:20.635015 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:49:20.636274 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:49:20.638176 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:49:20.638183 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:49:20.638185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:49:20.638494 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:49:20.638766 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:49:20.638767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:49:20.638767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:49:20.638900 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:49:20.639208 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:49:20.639215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:49:20.639366 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 03:49:20.639479 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:49:20.639634 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
I20260812 03:49:20.640275 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:20.630283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:44061 (local address 127.7.132.190:42717)
0812 03:49:20.630448 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:20.630450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:20.630483 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:49:20.631221 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:20.631222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:20.631233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:20.631299 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:49:20.632001 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:20.632504 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:20.633972 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:20.634168 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:20.634221 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:20.636365 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:49:20.636366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:49:20.636367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:49:20.636374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:49:20.638079 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:20.638600 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:20.638602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:20.638602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:20.638667 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:20.638994 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:20.638995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:20.638995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:20.639121 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:49:20.639200 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:20.640100 (+ 900us) server_negotiation.cc:299] Negotiation successful
0812 03:49:20.640185 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260812 03:49:20.640655 25876 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42717
I20260812 03:49:20.640743 25876 heartbeater.cc:464] Registering TS with master...
I20260812 03:49:20.640913 25876 heartbeater.cc:511] Master 127.7.132.190:42717 requested a full tablet report, sending...
I20260812 03:49:20.641446 25545 ts_manager.cc:194] Registered new tserver with Master: b4fec2785cbb41558b973dafd15c92f3 (127.7.132.129:42995)
I20260812 03:49:20.642318 25545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:44061
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:49:20.667522 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:33265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42717
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:49:20 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:49:20.795711 25890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:49:20.796047 25890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:49:20.796137 25890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:49:20.797806 25890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:49:20.797865 25890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:49:20.797907 25890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:49:20.800385 25890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.132.148:33265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:49:20.800738 25890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:49:20.801013 25890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:49:20.809429 25902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:49:20.809576 25901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:49:20.809880 25907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:49:20.810801 25890 server_base.cc:1034] running on GCE node
I20260812 03:49:20.810987 25890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:49:20.811219 25890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:49:20.812394 25890 hybrid_clock.cc:648] HybridClock initialized: now 1786506560812362 us; error 39 us; skew 500 ppm
Aug 12 03:49:20 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:49:20.815856 25890 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:49:20.816316 25890 webserver.cc:460] Webserver started at http://127.7.132.130:41427/ using document root <none> and password file <none>
I20260812 03:49:20.816565 25890 fs_manager.cc:359] Metadata directory not provided
I20260812 03:49:20.816610 25890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:49:20.816752 25890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:49:20 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:49:20.833035 25890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "0857663078404781902a20c55cf61fd6"
format_stamp: "Formatted at 2026-08-12 03:49:20 on dist-test-slave-39l8"
server_key: "08f745f7af3d5ff642f29917d1ab84ab"
server_key_iv: "f6f384cebe91dc72a461fd7bd29d4a34"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:20.833491 25890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "0857663078404781902a20c55cf61fd6"
format_stamp: "Formatted at 2026-08-12 03:49:20 on dist-test-slave-39l8"
server_key: "08f745f7af3d5ff642f29917d1ab84ab"
server_key_iv: "f6f384cebe91dc72a461fd7bd29d4a34"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:20.838858 25890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260812 03:49:20.859180 25916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:20.859447 25890 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260812 03:49:20.859571 25890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "0857663078404781902a20c55cf61fd6"
format_stamp: "Formatted at 2026-08-12 03:49:20 on dist-test-slave-39l8"
server_key: "08f745f7af3d5ff642f29917d1ab84ab"
server_key_iv: "f6f384cebe91dc72a461fd7bd29d4a34"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:20.859661 25890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:49:20.886631 25890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:49:20.889226 25890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:49:20.889423 25890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:49:20.889880 25890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:49:20.890375 25890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:49:20.890463 25890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:20.890545 25890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:49:20.890604 25890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:20.902872 25890 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:34987
I20260812 03:49:20.903442 25890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:49:20.904084 26076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:34987 every 8 connection(s)
I20260812 03:49:20.907019 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 25890
I20260812 03:49:20.907137 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:49:20.907315 7698 external_mini_cluster.cc:1428] Setting key 22dd6fdd851775dc68d8b33dfb81ae81
Aug 12 03:49:20 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:49:20.918859 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:20.904505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:41969 (local address 127.7.132.190:42717)
0812 03:49:20.904567 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:20.904569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:20.904941 (+ 372us) server_negotiation.cc:407] Connection header received
0812 03:49:20.905666 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:20.905668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:20.905685 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:20.905717 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:49:20.906586 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:20.907405 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:20.911601 (+ 4196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:20.911939 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:20.911983 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:20.914912 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:49:20.914914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:49:20.914914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:49:20.914922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:49:20.916879 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:20.917462 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:20.917463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:20.917464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:20.917527 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:20.917854 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:20.917856 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:20.917856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:20.918010 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:49:20.918079 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:20.918566 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 03:49:20.918681 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:49:20.919817 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:20.904515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42717 (local address 127.7.132.130:41969)
0812 03:49:20.904800 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:20.904849 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:49:20.905495 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:49:20.905901 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:49:20.905905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:49:20.906142 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:49:20.906433 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:20.906445 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:20.910152 (+ 3707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:20.910153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:20.911481 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:20.911484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:20.912114 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:20.912115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:20.912285 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:20.912876 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:49:20.912887 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:49:20.914717 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:49:20.916992 (+ 2275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:49:20.917000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:49:20.917003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:49:20.917358 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:49:20.917611 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:49:20.917612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:49:20.917613 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:49:20.917765 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:49:20.918140 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:49:20.918147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:49:20.918352 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 03:49:20.919340 (+ 988us) client_negotiation.cc:240] Negotiation successful
0812 03:49:20.919548 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":184,"threads_started":1}
I20260812 03:49:20.920609 26077 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42717
I20260812 03:49:20.920718 26077 heartbeater.cc:464] Registering TS with master...
I20260812 03:49:20.920938 26077 heartbeater.cc:511] Master 127.7.132.190:42717 requested a full tablet report, sending...
I20260812 03:49:20.921408 25545 ts_manager.cc:194] Registered new tserver with Master: 0857663078404781902a20c55cf61fd6 (127.7.132.130:34987)
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
I20260812 03:49:20.921905 25545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:41969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:49:20.939235 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:33265/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42717
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:49:21.057673 26085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:49:21.058037 26085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:49:21.058125 26085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:49:21.059859 26085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:49:21.059916 26085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:49:21.059953 26085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:49:21.062124 26085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.132.148:33265/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:49:21.062348 26085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:49:21.062677 26085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:49:21.069283 26093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:49:21.069293 26090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:49:21.069368 26098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:49:21.069708 26085 server_base.cc:1034] running on GCE node
I20260812 03:49:21.069943 26085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:49:21.070185 26085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:49:21.071377 26085 hybrid_clock.cc:648] HybridClock initialized: now 1786506561071354 us; error 39 us; skew 500 ppm
Aug 12 03:49:21 dist-test-slave-39l8 krb5kdc[24835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506561, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:49:21.074185 26085 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:49:21.074679 26085 webserver.cc:460] Webserver started at http://127.7.132.131:41827/ using document root <none> and password file <none>
I20260812 03:49:21.074978 26085 fs_manager.cc:359] Metadata directory not provided
I20260812 03:49:21.075037 26085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:49:21.075145 26085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:49:21 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506561, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:49:21.091499 26085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "e38de2d658834c11864f082bab8beda8"
format_stamp: "Formatted at 2026-08-12 03:49:21 on dist-test-slave-39l8"
server_key: "1b487e1aa413a94d44c1246b64569659"
server_key_iv: "dad7ce9d50e231e0e4686b4c4efd3e73"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:21.092003 26085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "e38de2d658834c11864f082bab8beda8"
format_stamp: "Formatted at 2026-08-12 03:49:21 on dist-test-slave-39l8"
server_key: "1b487e1aa413a94d44c1246b64569659"
server_key_iv: "dad7ce9d50e231e0e4686b4c4efd3e73"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:21.094134 26085 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 03:49:21.114143 26108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:21.114426 26085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:49:21.114603 26085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "e38de2d658834c11864f082bab8beda8"
format_stamp: "Formatted at 2026-08-12 03:49:21 on dist-test-slave-39l8"
server_key: "1b487e1aa413a94d44c1246b64569659"
server_key_iv: "dad7ce9d50e231e0e4686b4c4efd3e73"
server_key_version: "kuduclusterkey@0"
I20260812 03:49:21.114842 26085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:49:21.140360 26085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:49:21.142409 26085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:49:21.142583 26085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:49:21.142931 26085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:49:21.143307 26085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:49:21.143358 26085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:21.143422 26085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:49:21.143450 26085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:49:21.156666 26085 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:35059
I20260812 03:49:21.157193 26085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:49:21.159201 26298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:35059 every 8 connection(s)
I20260812 03:49:21.165571 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 26085
I20260812 03:49:21.165691 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:49:21.165876 7698 external_mini_cluster.cc:1428] Setting key 316254308e3983676eeb0e414e7cbc73
Aug 12 03:49:21 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506561, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:49:21.172505 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.158526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:60253 (local address 127.7.132.190:42717)
0812 03:49:21.158592 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:21.158595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:21.159937 (+ 1342us) server_negotiation.cc:407] Connection header received
0812 03:49:21.160797 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:21.160798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:21.160812 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:21.160891 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:49:21.161775 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.162517 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.163998 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.164333 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.164376 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.168458 (+ 4082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:49:21.168459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:49:21.168460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:49:21.168469 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:49:21.170227 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:21.170782 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:21.170784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:21.170784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:21.170862 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:21.171162 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:21.171163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:21.171163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:21.171299 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:49:21.171372 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:21.172313 (+ 941us) server_negotiation.cc:299] Negotiation successful
0812 03:49:21.172417 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:49:21.172652 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.158505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42717 (local address 127.7.132.131:60253)
0812 03:49:21.159801 (+ 1296us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:21.159850 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:49:21.160666 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:49:21.161022 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:49:21.161026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:49:21.161407 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:49:21.161666 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.161672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.162636 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.162638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.163889 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.163892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.164468 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.164469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.164669 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.166343 (+ 1674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:49:21.166353 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:49:21.168312 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:49:21.170330 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:49:21.170336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:49:21.170338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:49:21.170646 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:49:21.170938 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:49:21.170939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:49:21.170939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:49:21.171073 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:49:21.171930 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:49:21.171937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:49:21.172126 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 03:49:21.172243 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:49:21.172396 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"thread_start_us":244,"threads_started":1}
I20260812 03:49:21.173318 26300 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42717
I20260812 03:49:21.173455 26300 heartbeater.cc:464] Registering TS with master...
I20260812 03:49:21.173656 26300 heartbeater.cc:511] Master 127.7.132.190:42717 requested a full tablet report, sending...
I20260812 03:49:21.174078 25545 ts_manager.cc:194] Registered new tserver with Master: e38de2d658834c11864f082bab8beda8 (127.7.132.131:35059)
I20260812 03:49:21.174552 25545 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:60253
I20260812 03:49:21.177591 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:49:21.192800 25881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.183007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33348 (local address 127.7.132.190:42717)
0812 03:49:21.183068 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:21.183070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:21.183432 (+ 362us) server_negotiation.cc:407] Connection header received
0812 03:49:21.183483 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:21.183484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:21.183495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:21.183562 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:49:21.185377 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.186066 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.187170 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.187435 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.187509 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.189013 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:49:21.189014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:49:21.189015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:49:21.189022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:49:21.190681 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:21.191880 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:21.191881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:21.191881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:21.191942 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:49:21.192205 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:49:21.192206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:49:21.192206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:49:21.192329 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:49:21.192427 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:21.192637 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 03:49:21.192713 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:49:21.194855 25545 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:49:21.197248 25545 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:49:21.209940 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.204221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:35059 (local address 127.0.0.1:42112)
0812 03:49:21.205256 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:21.205301 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:49:21.205375 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:49:21.205699 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:49:21.205703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:49:21.205705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:49:21.205774 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.205776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.207223 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.207223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.208534 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.208537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.209657 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.209658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.209730 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.209742 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:49:21.209807 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:49:21.209852 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"thread_start_us":107,"threads_started":1}
I20260812 03:49:21.210418 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.204566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42112 (local address 127.7.132.131:35059)
0812 03:49:21.205033 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:21.205038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:21.205356 (+ 318us) server_negotiation.cc:407] Connection header received
0812 03:49:21.205490 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:21.205493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:21.205511 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:21.205604 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:49:21.205856 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.207140 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.208660 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.209478 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.209539 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.209628 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:21.210308 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 03:49:21.210346 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":89,"threads_started":1}
I20260812 03:49:21.211215 26190 tablet_service.cc:1467] Processing CreateTablet for tablet 07a3a6dc5e2044c4bffe130d5e874f1f (DEFAULT_TABLE table=test-table [id=8adb2ec0a1264a4799f663342158d397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:49:21.211621 26190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a3a6dc5e2044c4bffe130d5e874f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:49:21.211809 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.204221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34987 (local address 127.0.0.1:50164)
0812 03:49:21.205570 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:21.205580 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:49:21.205629 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:49:21.205912 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:49:21.205913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:49:21.205915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:49:21.205946 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.205947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.209554 (+ 3607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.209555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.210560 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.210564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.211590 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.211591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.211651 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.211655 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:49:21.211707 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:49:21.211745 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260812 03:49:21.213994 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.204713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50164 (local address 127.7.132.130:34987)
0812 03:49:21.205126 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:21.205131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:21.205687 (+ 556us) server_negotiation.cc:407] Connection header received
0812 03:49:21.205729 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:21.205731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:21.205746 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:21.205821 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:49:21.206022 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.207026 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.210693 (+ 3667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.211415 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.211483 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.211550 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:21.213867 (+ 2317us) server_negotiation.cc:299] Negotiation successful
0812 03:49:21.213913 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":188,"threads_started":1}
I20260812 03:49:21.214594 25987 tablet_service.cc:1467] Processing CreateTablet for tablet 07a3a6dc5e2044c4bffe130d5e874f1f (DEFAULT_TABLE table=test-table [id=8adb2ec0a1264a4799f663342158d397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:49:21.214927 25987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a3a6dc5e2044c4bffe130d5e874f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:49:21.217990 26334 tablet_bootstrap.cc:492] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: Bootstrap starting.
I20260812 03:49:21.218813 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.204456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42995 (local address 127.0.0.1:37262)
0812 03:49:21.207242 (+ 2786us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:21.207251 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:49:21.207296 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:49:21.207533 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:49:21.207534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:49:21.207535 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:49:21.207574 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.207575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.208764 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.208765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.212894 (+ 4129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.212897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.218516 (+ 5619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.218517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.218592 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.218597 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:49:21.218652 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:49:21.218719 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2769,"thread_start_us":73,"threads_started":1}
I20260812 03:49:21.219157 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.204857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37262 (local address 127.7.132.129:42995)
0812 03:49:21.205513 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:21.205518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:21.207352 (+ 1834us) server_negotiation.cc:407] Connection header received
0812 03:49:21.207387 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:21.207390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:21.207404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:21.207468 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:49:21.207644 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.208680 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.212996 (+ 4316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.213691 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.213756 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.213806 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:21.218998 (+ 5192us) server_negotiation.cc:299] Negotiation successful
0812 03:49:21.219062 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":60,"threads_started":1}
I20260812 03:49:21.219601 26334 tablet_bootstrap.cc:654] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: Neither blocks nor log segments found. Creating new log.
I20260812 03:49:21.220135 26334 log.cc:824] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:49:21.220163 25762 tablet_service.cc:1467] Processing CreateTablet for tablet 07a3a6dc5e2044c4bffe130d5e874f1f (DEFAULT_TABLE table=test-table [id=8adb2ec0a1264a4799f663342158d397]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:49:21.220513 25762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07a3a6dc5e2044c4bffe130d5e874f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:49:21.221024 26334 tablet_bootstrap.cc:492] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: No bootstrap required, opened a new log
I20260812 03:49:21.221082 26334 ts_tablet_manager.cc:1397] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:49:21.224185 26334 raft_consensus.cc:348] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.224359 26334 raft_consensus.cc:374] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:49:21.224396 26334 raft_consensus.cc:729] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0857663078404781902a20c55cf61fd6, State: Initialized, Role: FOLLOWER
I20260812 03:49:21.224535 26334 consensus_queue.cc:260] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.224853 26334 ts_tablet_manager.cc:1428] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 03:49:21.225005 26077 heartbeater.cc:503] Master 127.7.132.190:42717 was elected leader, sending a full tablet report...
I20260812 03:49:21.225780 26336 tablet_bootstrap.cc:492] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3: Bootstrap starting.
I20260812 03:49:21.226663 26336 tablet_bootstrap.cc:654] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3: Neither blocks nor log segments found. Creating new log.
I20260812 03:49:21.227033 26336 log.cc:824] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:49:21.227398 26333 tablet_bootstrap.cc:492] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Bootstrap starting.
I20260812 03:49:21.228199 26333 tablet_bootstrap.cc:654] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Neither blocks nor log segments found. Creating new log.
I20260812 03:49:21.228498 26333 log.cc:824] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:49:21.229101 26336 tablet_bootstrap.cc:492] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3: No bootstrap required, opened a new log
I20260812 03:49:21.229182 26336 ts_tablet_manager.cc:1397] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:49:21.230677 26333 tablet_bootstrap.cc:492] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: No bootstrap required, opened a new log
I20260812 03:49:21.230782 26333 ts_tablet_manager.cc:1397] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:49:21.232708 26333 raft_consensus.cc:348] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.232861 26333 raft_consensus.cc:374] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:49:21.234025 26336 raft_consensus.cc:348] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.234190 26336 raft_consensus.cc:374] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:49:21.234270 26333 raft_consensus.cc:729] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e38de2d658834c11864f082bab8beda8, State: Initialized, Role: FOLLOWER
I20260812 03:49:21.234288 26336 raft_consensus.cc:729] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fec2785cbb41558b973dafd15c92f3, State: Initialized, Role: FOLLOWER
I20260812 03:49:21.234458 26333 consensus_queue.cc:260] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.234457 26336 consensus_queue.cc:260] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.234793 26333 ts_tablet_manager.cc:1428] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 03:49:21.234793 26336 ts_tablet_manager.cc:1428] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260812 03:49:21.234963 26300 heartbeater.cc:503] Master 127.7.132.190:42717 was elected leader, sending a full tablet report...
I20260812 03:49:21.234963 25876 heartbeater.cc:503] Master 127.7.132.190:42717 was elected leader, sending a full tablet report...
Aug 12 03:49:21 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
W20260812 03:49:21.380342 25879 tablet.cc:2369] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:49:21.404824 26079 tablet.cc:2369] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:49:21.411125 26304 tablet.cc:2369] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:49:21.483270 26348 raft_consensus.cc:482] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:49:21.483418 26348 raft_consensus.cc:504] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.483950 26348 leader_election.cc:288] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0857663078404781902a20c55cf61fd6 (127.7.132.130:34987), b4fec2785cbb41558b973dafd15c92f3 (127.7.132.129:42995)
I20260812 03:49:21.487924 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.484256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34987 (local address 127.7.132.131:49213)
0812 03:49:21.484605 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:21.484616 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:49:21.484660 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:49:21.485151 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:49:21.485152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:49:21.485154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:49:21.485215 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.485217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.485974 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.485975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.486962 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.486964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.487687 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.487688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.487760 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.487764 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:49:21.487803 (+ 39us) client_negotiation.cc:240] Negotiation successful
0812 03:49:21.487837 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330}
I20260812 03:49:21.489413 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.484921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33177 (local address 127.7.132.129:42995)
0812 03:49:21.485120 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:21.485122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:21.485910 (+ 788us) server_negotiation.cc:407] Connection header received
0812 03:49:21.486004 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:21.486005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:21.486014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:21.486039 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:49:21.486345 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.486951 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.488426 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.489072 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.489117 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.489123 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:21.489313 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 03:49:21.489350 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260812 03:49:21.489609 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.484357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42995 (local address 127.7.132.131:33177)
0812 03:49:21.485830 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:21.485842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:49:21.485935 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:49:21.486097 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:49:21.486098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:49:21.486099 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:49:21.486269 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.486271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.487049 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.487050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.488294 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:49:21.488297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.489189 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.489190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:49:21.489247 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.489250 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:49:21.489484 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 03:49:21.489518 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1420,"mutex_wait_us":1,"thread_start_us":154,"threads_started":1}
I20260812 03:49:21.490448 25795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a3a6dc5e2044c4bffe130d5e874f1f" candidate_uuid: "e38de2d658834c11864f082bab8beda8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fec2785cbb41558b973dafd15c92f3" is_pre_election: true
I20260812 03:49:21.490622 25795 raft_consensus.cc:2454] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38de2d658834c11864f082bab8beda8 in term 0.
I20260812 03:49:21.491029 26109 leader_election.cc:302] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4fec2785cbb41558b973dafd15c92f3, e38de2d658834c11864f082bab8beda8; no voters:
I20260812 03:49:21.491323 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.484773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49213 (local address 127.7.132.130:34987)
0812 03:49:21.484966 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:21.484968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:21.484976 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:49:21.485005 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:21.485005 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:21.485014 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:21.485058 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:49:21.485291 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.485890 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.487113 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.487574 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.487652 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.487660 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:21.491226 (+ 3566us) server_negotiation.cc:299] Negotiation successful
0812 03:49:21.491267 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260812 03:49:21.491561 26348 raft_consensus.cc:2781] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:49:21.491551 26010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a3a6dc5e2044c4bffe130d5e874f1f" candidate_uuid: "e38de2d658834c11864f082bab8beda8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0857663078404781902a20c55cf61fd6" is_pre_election: true
I20260812 03:49:21.491629 26348 raft_consensus.cc:482] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:49:21.491652 26348 raft_consensus.cc:3037] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:49:21.491675 26010 raft_consensus.cc:2454] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e38de2d658834c11864f082bab8beda8 in term 0.
I20260812 03:49:21.492952 26348 raft_consensus.cc:504] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.493091 26348 leader_election.cc:288] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [CANDIDATE]: Term 1 election: Requested vote from peers 0857663078404781902a20c55cf61fd6 (127.7.132.130:34987), b4fec2785cbb41558b973dafd15c92f3 (127.7.132.129:42995)
I20260812 03:49:21.493288 26010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a3a6dc5e2044c4bffe130d5e874f1f" candidate_uuid: "e38de2d658834c11864f082bab8beda8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0857663078404781902a20c55cf61fd6"
I20260812 03:49:21.493323 25795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07a3a6dc5e2044c4bffe130d5e874f1f" candidate_uuid: "e38de2d658834c11864f082bab8beda8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fec2785cbb41558b973dafd15c92f3"
I20260812 03:49:21.493355 26010 raft_consensus.cc:3037] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:49:21.493391 25795 raft_consensus.cc:3037] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:49:21.494277 26010 raft_consensus.cc:2454] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38de2d658834c11864f082bab8beda8 in term 1.
I20260812 03:49:21.494469 25795 raft_consensus.cc:2454] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e38de2d658834c11864f082bab8beda8 in term 1.
I20260812 03:49:21.494526 26109 leader_election.cc:302] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0857663078404781902a20c55cf61fd6, e38de2d658834c11864f082bab8beda8; no voters:
I20260812 03:49:21.494593 26348 raft_consensus.cc:2781] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:49:21.494777 26348 raft_consensus.cc:686] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 1 LEADER]: Becoming Leader. State: Replica: e38de2d658834c11864f082bab8beda8, State: Running, Role: LEADER
I20260812 03:49:21.494859 26348 consensus_queue.cc:237] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } }
I20260812 03:49:21.495657 25545 catalog_manager.cc:5495] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e38de2d658834c11864f082bab8beda8 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "e38de2d658834c11864f082bab8beda8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e38de2d658834c11864f082bab8beda8" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 35059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:49:21 dist-test-slave-39l8 krb5kdc[24835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506560, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:49:21.567967 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.563914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51366 (local address 127.7.132.131:35059)
0812 03:49:21.563973 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:49:21.563975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:49:21.564165 (+ 190us) server_negotiation.cc:407] Connection header received
0812 03:49:21.564211 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:49:21.564212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:49:21.564233 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:49:21.564297 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:49:21.564596 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.565244 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.566520 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:49:21.566828 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:49:21.566929 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:49:21.567118 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:49:21.567246 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:49:21.567316 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:49:21.567870 (+ 554us) server_negotiation.cc:299] Negotiation successful
0812 03:49:21.567892 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:49:21.570559 25795 raft_consensus.cc:1261] T 07a3a6dc5e2044c4bffe130d5e874f1f P b4fec2785cbb41558b973dafd15c92f3 [term 1 FOLLOWER]: Refusing update from remote peer e38de2d658834c11864f082bab8beda8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:49:21.570575 26010 raft_consensus.cc:1261] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 1 FOLLOWER]: Refusing update from remote peer e38de2d658834c11864f082bab8beda8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:49:21.570890 26348 consensus_queue.cc:1035] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fec2785cbb41558b973dafd15c92f3" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:49:21.571081 26352 consensus_queue.cc:1035] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0857663078404781902a20c55cf61fd6" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:49:21.574076 26358 mvcc.cc:205] Tried to move back new op lower bound from 7317530876189249536 to 7317530875883401216. Current Snapshot: MvccSnapshot[applied={T|T < 7317530876189249536}]
I20260812 03:49:21.575699 26360 mvcc.cc:205] Tried to move back new op lower bound from 7317530876189249536 to 7317530875883401216. Current Snapshot: MvccSnapshot[applied={T|T < 7317530876189249536}]
I20260812 03:49:21.577175 25544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:49:21.577277 25544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:49:21.578193 25544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33ee7ac41a1942fd9ad29fc55d04bbe3: Sending DeleteTablet for 3 replicas of tablet 07a3a6dc5e2044c4bffe130d5e874f1f
I20260812 03:49:21.578560 26190 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a3a6dc5e2044c4bffe130d5e874f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:49:21 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:42112
I20260812 03:49:21.579319 26370 tablet_replica.cc:330] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: stopping tablet replica
I20260812 03:49:21.579416 26370 raft_consensus.cc:2229] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:49:21.579568 26370 raft_consensus.cc:2258] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:49:21.579699 25762 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a3a6dc5e2044c4bffe130d5e874f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:49:21 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:37262
I20260812 03:49:21.580078 25987 tablet_service.cc:1514] Processing DeleteTablet for tablet 07a3a6dc5e2044c4bffe130d5e874f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:49:21 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:50164
I20260812 03:49:21.580492 26370 ts_tablet_manager.cc:1905] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:49:21.580806 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 25665
I20260812 03:49:21.589370 26370 ts_tablet_manager.cc:1918] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:49:21.589712 26370 log.cc:1196] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/07a3a6dc5e2044c4bffe130d5e874f1f
I20260812 03:49:21.590883 26370 ts_tablet_manager.cc:1939] T 07a3a6dc5e2044c4bffe130d5e874f1f P e38de2d658834c11864f082bab8beda8: Deleting consensus metadata
I20260812 03:49:21.591329 25527 catalog_manager.cc:4841] TS e38de2d658834c11864f082bab8beda8 (127.7.132.131:35059): tablet 07a3a6dc5e2044c4bffe130d5e874f1f (table test-table [id=8adb2ec0a1264a4799f663342158d397]) successfully deleted
I20260812 03:49:21.595219 26359 mvcc.cc:205] Tried to move back new op lower bound from 7317530876189249536 to 7317530875883401216. Current Snapshot: MvccSnapshot[applied={T|T < 7317530876189249536}]
I20260812 03:49:21.602921 26373 tablet_replica.cc:330] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: stopping tablet replica
I20260812 03:49:21.603125 26373 raft_consensus.cc:2229] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:49:21.603307 26373 raft_consensus.cc:2258] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:49:21.603921 26373 ts_tablet_manager.cc:1905] T 07a3a6dc5e2044c4bffe130d5e874f1f P 0857663078404781902a20c55cf61fd6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:49:21.606108 25522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:42995 (error 108)
I20260812 03:49:21.606346 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 25890
I20260812 03:49:21.642128 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.641981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42995 (local address 127.0.0.1:39062)
0812 03:49:21.642039 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:21.642088 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:49:21.642424 25522 catalog_manager.cc:4584] TS b4fec2785cbb41558b973dafd15c92f3 (127.7.132.129:42995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07a3a6dc5e2044c4bffe130d5e874f1f: Network error: Client connection negotiation failed: client connection to 127.7.132.129:42995: connect: Connection refused (error 111)
I20260812 03:49:21.643518 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:49:21.643437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34987 (local address 127.0.0.1:52370)
0812 03:49:21.643474 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 03:49:21.643494 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.130:34987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 03:49:21.643611 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 26085
I20260812 03:49:21.666199 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 25502
I20260812 03:49:21.687155 7698 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:49:22.031689 7698 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:49:22.031821 7698 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:49:22.377388 7698 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:49:22.377 UTC [24890] LOG: received smart shutdown request
2026-08-12 03:49:22.381 UTC [24890] LOG: background worker "logical replication launcher" (PID 24901) exited with exit code 1
2026-08-12 03:49:22.381 UTC [24894] LOG: shutting down
2026-08-12 03:49:22.514 UTC [24890] LOG: database system is shut down
2026-08-12T03:49:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45068 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:49:22 dist-test-slave-39l8 krb5kdc[26392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:49:22 dist-test-slave-39l8 krb5kdc[26392](info): set up 2 sockets
Aug 12 03:49:22 dist-test-slave-39l8 krb5kdc[26392](info): commencing operation
krb5kdc: starting...
W20260812 03:49:23.821982 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:49:23 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:49:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/rangeradmin_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/rangeradmin_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/rangerlookup_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/rangerlookup_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
I20260812 03:49:23.971141 7698 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskWjyozu/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/postgres -l logfile start
2026-08-12 03:49:26.114 UTC [26429] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:49:26.114 UTC [26429] LOG: listening on IPv4 address "127.7.132.148", port 35899
2026-08-12 03:49:26.117 UTC [26429] LOG: listening on Unix socket "/tmp/.s.PGSQL.35899"
2026-08-12 03:49:26.137 UTC [26433] LOG: database system was shut down at 2026-08-12 03:49:24 UTC
2026-08-12 03:49:26.140 UTC [26429] LOG: database system is ready to accept connections
I20260812 03:49:27.252110 7698 mini_postgres.cc:96] Postgres bound to 35899
2026-08-12 03:49:27.258 UTC [26450] FATAL: database "slave" does not exist
127.7.132.148:35899 - accepting connections
I20260812 03:49:27.259415 7698 mini_ranger.cc:161] Starting Ranger...
I20260812 03:49:27.273721 7698 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:49:27.663447 7698 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:49:27.663534 7698 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-admin
2026-08-12 03:49:27,927 [I] DB FLAVOR :POSTGRES
2026-08-12 03:49:27,927 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:49:27,927 [I] Checking connection..
2026-08-12 03:49:27,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:49:28,184 [I] Checking connection passed.
2026-08-12 03:49:28,184 [I] --------- Verifying version history table ---------
2026-08-12 03:49:28,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:49:28,402 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:49:28,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:49:28,657 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:49:28,657 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:49:28,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:49:28.900 UTC [26561] WARNING: there is no transaction in progress
2026-08-12 03:49:28,908 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:49:28,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:49:29,143 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:49:29,143 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:49:29,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:49:29,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:49:29,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-39l8.c.gcp-upstream.internal','N') ;"
2026-08-12 03:49:29,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:49:30,085 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:49:30,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:49:30,321 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:49:30,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:49:30,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:49:30,774 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:49:30,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:49:31.637 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.651 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.666 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.679 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.693 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.711 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.761 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.767 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.779 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.788 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.797 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.809 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.817 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.822 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.828 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:31.833 UTC [26736] WARNING: there is no transaction in progress
2026-08-12 03:49:32,375 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:49:32,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:49:32,619 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:49:32,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:49:32,838 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:49:32,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:49:33,060 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:49:33,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:49:33,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:49:33,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-39l8.c.gcp-upstream.internal';"
2026-08-12 03:49:33,777 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:49:33,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:49:34,020 [I] DB_PATCHES have already been applied
I20260812 03:49:34.028033 7698 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-admin:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskWjyozu/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:49:34.028091 7698 mini_ranger.cc:193] Using host: 127.7.132.148
I20260812 03:49:34.032104 7698 mini_ranger.cc:227] Ranger admin URL: http://127.7.132.148:40359
Aug 12, 2026 3:49:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:49:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:49:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:49:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:49:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:49:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:49:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:49:35.337904 7698 mini_ranger.cc:160] Time spent starting Ranger: real 8.079s user 0.006s sys 0.004s
Aug 12, 2026 3:49:54 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:49:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 3:49:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:49:54 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:49:55 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:49:55.788638 7698 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.132.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.132.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/rangerkms_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.132.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/rangerkms_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.132.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.132.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.132.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/HTTP_127.7.132.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:49:55 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506595, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:49:55.865387 7698 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:49:55.874513 7698 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:49:56.598070 7698 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:49:56.598165 7698 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms
I20260812 03:49:56.598186 7698 mini_ranger_kms.cc:227] Using postgres at 127.7.132.148:35899
2026-08-12 03:49:56,761 [I] DB FLAVOR :POSTGRES
2026-08-12 03:49:56,761 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:49:56,762 [I] Checking connection
2026-08-12 03:49:56,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:49:56,993 [I] connection success
2026-08-12 03:49:56,993 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:49:56,993 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:49:56,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:49:57,246 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:49:57,246 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:49:57,246 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:49:57,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.132.148:35899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:49:57,519 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:49:57.974457 7698 mini_ranger_kms.cc:310] Created kms service
I20260812 03:49:58.182905 7698 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:49:58.234727 7698 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:49:58 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506595, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:49:58.595005 7698 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:49:58.595106 7698 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms:/tmp/dist-test-taskWjyozu/build/release/bin/postgresql.jar:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskWjyozu/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:49:58.595124 7698 mini_ranger_kms.cc:240] Using host: 127.7.132.148
I20260812 03:49:58.599582 7698 mini_ranger_kms.cc:276] Ranger KMS PID: 27005
I20260812 03:49:58.599654 7698 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.132.148:56579
Aug 12, 2026 3:49:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:49:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:49:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:49:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:50:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:50:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:50:00 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506600, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:01.228019 7698 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.363s user 0.003s sys 0.146s
I20260812 03:50:01.228154 7698 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:50:01.228204 7698 mini_ranger_kms.cc:401] 127.7.132.148:56579/kms/v1/keys
Aug 12 03:50:01 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506600, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Aug 12 03:50:01 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506600, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.132.148@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
Aug 12 03:50:03 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:50:03.859304 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33207
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33207
--ranger_config_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:56579/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:03.957157 27075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:03.957342 27075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:03.957363 27075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:03.958454 27075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:03.958492 27075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:03.958505 27075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:03.958516 27075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:03.958527 27075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:03.960191 27075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.132.148:56579/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33207
--ranger_config_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:03.960446 27075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:03.960690 27075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:50:03.965718 27075 server_base.cc:1034] running on GCE node
W20260812 03:50:03.965728 27092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:03.965701 27090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:03.965991 27086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:03.966236 27075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:03.966519 27075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:03.967680 27075 hybrid_clock.cc:648] HybridClock initialized: now 1786506603967647 us; error 52 us; skew 500 ppm
Aug 12 03:50:03 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506603, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:03.974064 27075 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:03.974643 27075 webserver.cc:460] Webserver started at http://127.7.132.190:42659/ using document root <none> and password file <none>
I20260812 03:50:03.974933 27075 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:03.974998 27075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:03.975145 27075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:50:03 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506603, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:04.025013 27075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "e0d79e01cf5f45b3b0585c3c56cb3018"
format_stamp: "Formatted at 2026-08-12 03:50:04 on dist-test-slave-39l8"
server_key: "5de55aa7b840f4d076dc7b0fa359a302"
server_key_iv: "c19223a87be83ca0e7f8764fb052e03a"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:04.025523 27075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "e0d79e01cf5f45b3b0585c3c56cb3018"
format_stamp: "Formatted at 2026-08-12 03:50:04 on dist-test-slave-39l8"
server_key: "5de55aa7b840f4d076dc7b0fa359a302"
server_key_iv: "c19223a87be83ca0e7f8764fb052e03a"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:04.027467 27075 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260812 03:50:04.041831 27103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:04.042037 27075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:04.042121 27075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "e0d79e01cf5f45b3b0585c3c56cb3018"
format_stamp: "Formatted at 2026-08-12 03:50:04 on dist-test-slave-39l8"
server_key: "5de55aa7b840f4d076dc7b0fa359a302"
server_key_iv: "c19223a87be83ca0e7f8764fb052e03a"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:04.042217 27075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:04.054347 27075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:04.056381 27075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:04.056524 27075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:04.061419 27075 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:33207
I20260812 03:50:04.061484 27186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:33207 every 8 connection(s)
I20260812 03:50:04.061841 27075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:04.062578 27187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:04.064785 27187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Bootstrap starting.
I20260812 03:50:04.065405 27187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:04.065728 27187 log.cc:824] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:04.066457 27187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: No bootstrap required, opened a new log
I20260812 03:50:04.066772 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 27075
I20260812 03:50:04.066861 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:50:04.067041 7698 external_mini_cluster.cc:1428] Setting key 77cf708d926adefa5cf6512589738928
I20260812 03:50:04.068863 27187 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } }
I20260812 03:50:04.069023 27187 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:04.069087 27187 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d79e01cf5f45b3b0585c3c56cb3018, State: Initialized, Role: FOLLOWER
I20260812 03:50:04.069265 27187 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } }
I20260812 03:50:04.069361 27187 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:50:04.069415 27187 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:50:04.069484 27187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:04.070039 27187 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } }
I20260812 03:50:04.070159 27187 leader_election.cc:302] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0d79e01cf5f45b3b0585c3c56cb3018; no voters:
I20260812 03:50:04.070353 27187 leader_election.cc:288] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:50:04.070380 27192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:04.070573 27192 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 1 LEADER]: Becoming Leader. State: Replica: e0d79e01cf5f45b3b0585c3c56cb3018, State: Running, Role: LEADER
I20260812 03:50:04.070645 27187 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:04.070693 27192 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } }
I20260812 03:50:04.071115 27192 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } } }
I20260812 03:50:04.071468 27192 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:04.071679 27192 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0d79e01cf5f45b3b0585c3c56cb3018. Latest consensus state: current_term: 1 leader_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } } }
I20260812 03:50:04.071826 27192 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:04.072162 27187 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:50:04.072780 27210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:04.073287 27210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:50:04 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:04.079160 27210 catalog_manager.cc:1347] Generated new cluster ID: 2ff85b8cb3fc4b968dbb2d4edeffff3b
I20260812 03:50:04.079281 27210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:50:04.082199 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:04.067543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.7.132.190:33207)
0812 03:50:04.067792 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:04.067801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:04.067831 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:50:04.068873 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:04.068883 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:04.068949 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:04.069886 (+ 937us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:04.069997 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:04.071373 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:04.072474 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:04.073056 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:04.073455 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:04.076342 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:04.076401 (+ 59us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:04.076401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:04.076509 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:04.079434 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:04.079895 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:04.079898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:04.079898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:04.080014 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:04.080222 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:04.080223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:04.080223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:04.080566 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:04.080684 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:04.081165 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 03:50:04.081536 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":63,"threads_started":1}
I20260812 03:50:04.087927 27210 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:50:04.088701 27210 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:50:04.096738 27210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Generated new TSK 0
I20260812 03:50:04.096954 27210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:50:04.756989 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:56579/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33207
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:04.859405 27257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:04.859611 27257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:04.859644 27257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:04.861120 27257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:04.861182 27257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:04.861218 27257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:50:04.863312 27257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.132.148:56579/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:04.863550 27257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:04.863830 27257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:04.869737 27268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:04.869750 27266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:04.870081 27264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:04.870589 27257 server_base.cc:1034] running on GCE node
I20260812 03:50:04.870841 27257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:04.871137 27257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:04.872330 27257 hybrid_clock.cc:648] HybridClock initialized: now 1786506604872319 us; error 32 us; skew 500 ppm
Aug 12 03:50:04 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506604, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:04.875138 27257 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:50:04.875669 27257 webserver.cc:460] Webserver started at http://127.7.132.129:42915/ using document root <none> and password file <none>
I20260812 03:50:04.875939 27257 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:04.876010 27257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:04.876145 27257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:50:04 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506604, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:04.889871 27257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e"
format_stamp: "Formatted at 2026-08-12 03:50:04 on dist-test-slave-39l8"
server_key: "ee25fe182162121e934ab44093546d3a"
server_key_iv: "3730ee269f10073fdccd373f76c84c97"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:04.890302 27257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e"
format_stamp: "Formatted at 2026-08-12 03:50:04 on dist-test-slave-39l8"
server_key: "ee25fe182162121e934ab44093546d3a"
server_key_iv: "3730ee269f10073fdccd373f76c84c97"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:04.893092 27257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 03:50:04.914891 27280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:04.915191 27257 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260812 03:50:04.915349 27257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e"
format_stamp: "Formatted at 2026-08-12 03:50:04 on dist-test-slave-39l8"
server_key: "ee25fe182162121e934ab44093546d3a"
server_key_iv: "3730ee269f10073fdccd373f76c84c97"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:04.915527 27257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:04.935639 27257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:04.938006 27257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:04.938196 27257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:04.938476 27257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:04.938923 27257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:04.939008 27257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:04.939098 27257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:04.939165 27257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:04.946822 27257 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:33965
I20260812 03:50:04.947378 27257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:50:04.949015 27452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:33965 every 8 connection(s)
I20260812 03:50:04.954084 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 27257
I20260812 03:50:04.954187 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:50:04.954393 7698 external_mini_cluster.cc:1428] Setting key c40fd4320b483834b9609e6ab97e4710
Aug 12 03:50:04 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506604, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:04.960480 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:04.949921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33207 (local address 127.7.132.129:56091)
0812 03:50:04.950101 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:04.950142 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:04.950795 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:04.951217 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:04.951222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:04.951458 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:04.951744 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:04.951751 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:04.952811 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:04.952812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:04.954046 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:04.954049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:04.954602 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:04.954603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:04.954762 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:04.955309 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:04.955318 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:04.956833 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:04.958824 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:04.958831 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:04.958833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:04.959102 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:04.959364 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:04.959365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:04.959365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:04.959496 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:04.959818 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:04.959825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:04.959980 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 03:50:04.960088 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:50:04.960253 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260812 03:50:04.960827 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:04.949928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:56091 (local address 127.7.132.190:33207)
0812 03:50:04.950980 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:04.950982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:04.950990 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:50:04.951017 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:04.951018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:04.951031 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:04.951092 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:04.951875 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:04.952689 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:04.954144 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:04.954465 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:04.954525 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:04.956934 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:04.956936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:04.956936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:04.956943 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:04.958734 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:04.959200 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:04.959201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:04.959202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:04.959276 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:04.959582 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:04.959583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:04.959583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:04.959719 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:04.959786 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:04.960627 (+ 841us) server_negotiation.cc:299] Negotiation successful
0812 03:50:04.960715 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":90,"threads_started":1}
I20260812 03:50:04.961382 27453 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33207
I20260812 03:50:04.961483 27453 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:04.961673 27453 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:04.962296 27133 ts_manager.cc:194] Registered new tserver with Master: 43c9d19d20e9497faaf9f98ea5cb4e7e (127.7.132.129:33965)
I20260812 03:50:04.963202 27133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:56091
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:50:05.009855 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:56579/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33207
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:50:05.127658 27464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:05.127868 27464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:05.127905 27464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:05.129040 27464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:05.129096 27464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:05.129149 27464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:50:05.130573 27464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.132.148:56579/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:05.130843 27464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:05.131085 27464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:05.135911 27474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:05.135906 27470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:05.135902 27469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:05.136094 27464 server_base.cc:1034] running on GCE node
I20260812 03:50:05.136330 27464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:05.136538 27464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:05.137689 27464 hybrid_clock.cc:648] HybridClock initialized: now 1786506605137661 us; error 42 us; skew 500 ppm
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:05.140283 27464 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:50:05.140756 27464 webserver.cc:460] Webserver started at http://127.7.132.130:33379/ using document root <none> and password file <none>
I20260812 03:50:05.140998 27464 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:05.141041 27464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:05.141202 27464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:05.154238 27464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "594ad7fdfb1446848366b404620d4f98"
format_stamp: "Formatted at 2026-08-12 03:50:05 on dist-test-slave-39l8"
server_key: "c26550785eb1234bb754f66982f58199"
server_key_iv: "2737928e0fb3fb2762c4d7a4c1c9918c"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:05.154793 27464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "594ad7fdfb1446848366b404620d4f98"
format_stamp: "Formatted at 2026-08-12 03:50:05 on dist-test-slave-39l8"
server_key: "c26550785eb1234bb754f66982f58199"
server_key_iv: "2737928e0fb3fb2762c4d7a4c1c9918c"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:05.157487 27464 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:50:05.169646 27485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.169840 27464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:05.169943 27464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "594ad7fdfb1446848366b404620d4f98"
format_stamp: "Formatted at 2026-08-12 03:50:05 on dist-test-slave-39l8"
server_key: "c26550785eb1234bb754f66982f58199"
server_key_iv: "2737928e0fb3fb2762c4d7a4c1c9918c"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:05.170042 27464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:05.187469 27464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:05.189507 27464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:05.189718 27464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:05.190011 27464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:05.190392 27464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:05.190445 27464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.190511 27464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:05.190539 27464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.197290 27464 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:40787
I20260812 03:50:05.197753 27464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:50:05.199185 27658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:40787 every 8 connection(s)
I20260812 03:50:05.200276 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 27464
I20260812 03:50:05.200390 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:50:05.200594 7698 external_mini_cluster.cc:1428] Setting key e84f7a52749b09619d7edc43a8dfabb3
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:05.211875 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:05.199837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33207 (local address 127.7.132.130:55845)
0812 03:50:05.202829 (+ 2992us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:05.202880 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:05.203500 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:05.203773 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:05.203777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:05.203975 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:05.204156 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:05.204160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.204974 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.204975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:05.205768 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:05.205770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.206251 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.206252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:05.206380 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:05.206960 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:05.206968 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:05.208545 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:05.210248 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:05.210254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:05.210257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:05.210475 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:05.210691 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:05.210691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:05.210692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:05.210872 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:05.211175 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:05.211181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:05.211342 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 03:50:05.211436 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:50:05.211592 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2945,"thread_start_us":83,"threads_started":1}
I20260812 03:50:05.212244 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:05.199828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:55845 (local address 127.7.132.190:33207)
0812 03:50:05.199881 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:05.199884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:05.202974 (+ 3090us) server_negotiation.cc:407] Connection header received
0812 03:50:05.203613 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:05.203614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:05.203626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:05.203695 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:05.204255 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.204860 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.205863 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.206083 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.206238 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:05.208664 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:05.208666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:05.208666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:05.208675 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:05.210138 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:05.210563 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:05.210564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:05.210565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:05.210623 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:05.210951 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:05.210952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:05.210953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:05.211091 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:05.211148 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:05.212063 (+ 915us) server_negotiation.cc:299] Negotiation successful
0812 03:50:05.212159 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:50:05.212618 27660 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33207
I20260812 03:50:05.212708 27660 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:05.212914 27660 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:05.213383 27133 ts_manager.cc:194] Registered new tserver with Master: 594ad7fdfb1446848366b404620d4f98 (127.7.132.130:40787)
I20260812 03:50:05.213914 27133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:55845
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:50:05.233541 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:56579/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33207
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:05.344749 27670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:05.344969 27670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:05.345007 27670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:05.346264 27670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:05.346324 27670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:05.346366 27670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:50:05.348034 27670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.132.148:56579/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:05.348279 27670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:05.348537 27670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:05.353597 27677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:05.353586 27678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:05.353729 27681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:05.353964 27670 server_base.cc:1034] running on GCE node
I20260812 03:50:05.354157 27670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:05.354441 27670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:05.355611 27670 hybrid_clock.cc:648] HybridClock initialized: now 1786506605355603 us; error 47 us; skew 500 ppm
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:05.358647 27670 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:50:05.359053 27670 webserver.cc:460] Webserver started at http://127.7.132.131:43995/ using document root <none> and password file <none>
I20260812 03:50:05.359297 27670 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:05.359362 27670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:05.359511 27670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:05.380927 27670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "f4aec3e0d0bc4733bf7c887400c01d01"
format_stamp: "Formatted at 2026-08-12 03:50:05 on dist-test-slave-39l8"
server_key: "eeefc9db1b352d024267b9bd72b2e161"
server_key_iv: "f4314584585d81b94c5b9ab0f9c910d5"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:05.381462 27670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "f4aec3e0d0bc4733bf7c887400c01d01"
format_stamp: "Formatted at 2026-08-12 03:50:05 on dist-test-slave-39l8"
server_key: "eeefc9db1b352d024267b9bd72b2e161"
server_key_iv: "f4314584585d81b94c5b9ab0f9c910d5"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:05.384639 27670 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:50:05.402644 27692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.402907 27670 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260812 03:50:05.403002 27670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "f4aec3e0d0bc4733bf7c887400c01d01"
format_stamp: "Formatted at 2026-08-12 03:50:05 on dist-test-slave-39l8"
server_key: "eeefc9db1b352d024267b9bd72b2e161"
server_key_iv: "f4314584585d81b94c5b9ab0f9c910d5"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:05.403069 27670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:05.418641 27670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:05.420886 27670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:05.421065 27670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:05.421304 27670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:05.421725 27670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:05.421797 27670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.421833 27670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:05.421861 27670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.429584 27670 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:41145
I20260812 03:50:05.430066 27670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:50:05.431090 27877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:41145 every 8 connection(s)
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:05.439879 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:05.431713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:58561 (local address 127.7.132.190:33207)
0812 03:50:05.431824 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:05.431827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:05.432206 (+ 379us) server_negotiation.cc:407] Connection header received
0812 03:50:05.432752 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:05.432753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:05.432766 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:05.432820 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:05.433378 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.433849 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.434757 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.434947 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.435016 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:05.436925 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:05.436927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:05.436927 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:05.436935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:05.438293 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:05.438794 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:05.438807 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:05.438807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:05.438882 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:05.439207 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:05.439208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:05.439208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:05.439340 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:05.439427 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:05.439699 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 03:50:05.439787 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:50:05.440017 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:05.431694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33207 (local address 127.7.132.131:58561)
0812 03:50:05.432074 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:05.432105 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:05.432633 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:05.432892 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:05.432896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:05.433078 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:05.433282 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:05.433286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.433942 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.433943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:05.434647 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:05.434650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.435050 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.435051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:05.435162 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:05.435612 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:05.435620 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:05.436831 (+ 1211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:05.438389 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:05.438396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:05.438398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:05.438674 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:05.438950 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:05.438951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:05.438951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:05.439106 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:05.439431 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:05.439438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:05.439556 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 03:50:05.439656 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:50:05.439792 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":119,"threads_started":1}
I20260812 03:50:05.440246 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 27670
I20260812 03:50:05.440348 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:50:05.440459 27879 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33207
I20260812 03:50:05.440539 7698 external_mini_cluster.cc:1428] Setting key c4c5e3f1311f0728684d93975898cb4b
I20260812 03:50:05.440543 27879 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:05.440721 27879 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:05.441111 27133 ts_manager.cc:194] Registered new tserver with Master: f4aec3e0d0bc4733bf7c887400c01d01 (127.7.132.131:41145)
I20260812 03:50:05.441429 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:50:05.441541 27133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:58561
I20260812 03:50:05.442180 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 27257
I20260812 03:50:05.452631 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 27464
I20260812 03:50:05.479632 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 27670
I20260812 03:50:05.493549 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 27075
I20260812 03:50:05.506475 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33207
--webserver_interface=127.7.132.190
--webserver_port=42659
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33207
--ranger_config_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:56579/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:56579/kms with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:05.585747 27882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:05.585974 27882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:05.586012 27882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:05.587138 27882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:05.587205 27882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:05.587226 27882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:05.587246 27882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:05.587286 27882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:05.588814 27882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:56579/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33207
--ranger_config_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=42659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:05.589097 27882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:05.589304 27882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:50:05.594029 27882 server_base.cc:1034] running on GCE node
W20260812 03:50:05.593918 27899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:05.593935 27892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:05.593919 27897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:05.594430 27882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:05.594684 27882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:05.595877 27882 hybrid_clock.cc:648] HybridClock initialized: now 1786506605595868 us; error 27 us; skew 500 ppm
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:05.598192 27882 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:05.598634 27882 webserver.cc:460] Webserver started at http://127.7.132.190:42659/ using document root <none> and password file <none>
I20260812 03:50:05.598901 27882 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:05.598956 27882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:05.643946 27882 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:05.644783 27910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.644974 27882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:05.645092 27882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "e0d79e01cf5f45b3b0585c3c56cb3018"
format_stamp: "Formatted at 2026-08-12 03:50:04 on dist-test-slave-39l8"
server_key: "5de55aa7b840f4d076dc7b0fa359a302"
server_key_iv: "c19223a87be83ca0e7f8764fb052e03a"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:05.645572 27882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:05.655037 27882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:05.657536 27882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:05.657792 27882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:05.663916 27882 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:33207
I20260812 03:50:05.663950 27989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:33207 every 8 connection(s)
I20260812 03:50:05.664439 27882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:05.664636 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 27882
I20260812 03:50:05.664934 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:56579/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:33965
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:33207
--webserver_port=42915
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:56579/kms with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:05.666309 27990 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:50:05.667196 27990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Bootstrap starting.
I20260812 03:50:05.676070 27990 log.cc:824] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:05.678138 27990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:05.678459 27990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Bootstrap complete.
I20260812 03:50:05.681741 27990 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } }
I20260812 03:50:05.682090 27990 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d79e01cf5f45b3b0585c3c56cb3018, State: Initialized, Role: FOLLOWER
I20260812 03:50:05.682286 27990 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } }
I20260812 03:50:05.682361 27990 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:50:05.682430 27990 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:50:05.682480 27990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:50:05.683811 27990 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } }
I20260812 03:50:05.683911 27990 leader_election.cc:302] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0d79e01cf5f45b3b0585c3c56cb3018; no voters:
I20260812 03:50:05.684103 27990 leader_election.cc:288] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:50:05.684165 28000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:50:05.684353 28000 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [term 2 LEADER]: Becoming Leader. State: Replica: e0d79e01cf5f45b3b0585c3c56cb3018, State: Running, Role: LEADER
I20260812 03:50:05.684428 27990 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:05.684484 28000 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } }
I20260812 03:50:05.684779 28000 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } } }
I20260812 03:50:05.684890 28000 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:05.685042 28000 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0d79e01cf5f45b3b0585c3c56cb3018. Latest consensus state: current_term: 2 leader_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d79e01cf5f45b3b0585c3c56cb3018" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33207 } } }
I20260812 03:50:05.685122 28000 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:05.685916 27990 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:50:05.689499 28011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:05.690217 28011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:50:05.690636 28011 catalog_manager.cc:1259] Loaded cluster ID: 2ff85b8cb3fc4b968dbb2d4edeffff3b
I20260812 03:50:05.690768 28011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:50:05.692786 28011 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:50:05.693065 28011 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Loaded TSK: 0
I20260812 03:50:05.693414 28011 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:50:05.776585 27995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:05.776916 27995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:05.777002 27995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:05.778570 27995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:05.778674 27995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:05.778772 27995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:50:05.780916 27995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:56579/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:33965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=42915
--tserver_master_addrs=127.7.132.190:33207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:05.781472 27995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:05.781841 27995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:05.788472 28048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:05.788470 28045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:05.789551 27995 server_base.cc:1034] running on GCE node
W20260812 03:50:05.794199 28044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:05.794942 27995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:05.795213 27995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:05.796389 27995 hybrid_clock.cc:648] HybridClock initialized: now 1786506605796366 us; error 38 us; skew 500 ppm
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:05.799664 27995 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:50:05.800159 27995 webserver.cc:460] Webserver started at http://127.7.132.129:42915/ using document root <none> and password file <none>
I20260812 03:50:05.800395 27995 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:05.800460 27995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:05.819128 27995 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:50:05.820377 28058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.820586 27995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:50:05.820676 27995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e"
format_stamp: "Formatted at 2026-08-12 03:50:04 on dist-test-slave-39l8"
server_key: "ee25fe182162121e934ab44093546d3a"
server_key_iv: "3730ee269f10073fdccd373f76c84c97"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:05.821144 27995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:05.839408 27995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:05.846844 27995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:05.847011 27995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:05.847242 27995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:05.847680 27995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:05.847730 27995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.847766 27995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:05.847790 27995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:05.852591 27995 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:33965
I20260812 03:50:05.853063 27995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:50:05.855211 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 27995
I20260812 03:50:05.855471 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:56579/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:40787
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:33207
--webserver_port=33379
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:56579/kms with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:05.860721 28220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:33965 every 8 connection(s)
Aug 12 03:50:05 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506605, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:05.877671 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:05.854111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33207 (local address 127.7.132.129:51445)
0812 03:50:05.854738 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:05.854783 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:05.858427 (+ 3644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:05.859042 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:05.859046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:05.859245 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:05.859487 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:05.859492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.861010 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.861011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:05.862188 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:05.862191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.863219 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.863220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:05.863358 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:05.864988 (+ 1630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:05.865000 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:05.866598 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:05.875775 (+ 9177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:05.875782 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:05.875785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:05.876089 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:05.876367 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:05.876368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:05.876368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:05.876487 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:05.876991 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:05.876998 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:05.877143 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 03:50:05.877250 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:50:05.877402 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":75,"threads_started":1}
I20260812 03:50:05.878432 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:05.854495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:51445 (local address 127.7.132.190:33207)
0812 03:50:05.857516 (+ 3021us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:05.857525 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:05.857564 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:50:05.858736 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:05.858742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:05.858795 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:05.858906 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:05.859607 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.860893 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.862293 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:05.862986 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:05.863094 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:05.866782 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:05.866787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:05.866787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:05.866799 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:05.875668 (+ 8869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:05.876187 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:05.876190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:05.876191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:05.876298 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:05.876560 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:05.876561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:05.876562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:05.876876 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:05.876950 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:05.878110 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0812 03:50:05.878242 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2965,"thread_start_us":88,"threads_started":1}
I20260812 03:50:05.879644 28223 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33207
I20260812 03:50:05.879755 28223 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:05.879942 28223 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
W20260812 03:50:05.880913 28223 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33207 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:50:05.880978 28223 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:05.881037 28223 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:05.881433 28223 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:05.881505 28223 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
W20260812 03:50:05.881932 28223 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:50:05.989382 28229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:05.989705 28229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:05.989790 28229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:05.992471 28229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:05.992580 28229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:05.992661 28229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:50:05.994561 28229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:56579/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:40787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=33379
--tserver_master_addrs=127.7.132.190:33207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:05.995066 28229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:05.995340 28229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:06.001039 28242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:06.001379 28246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:06.001896 28237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:06.002096 28229 server_base.cc:1034] running on GCE node
I20260812 03:50:06.002470 28229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:06.002816 28229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:06.004006 28229 hybrid_clock.cc:648] HybridClock initialized: now 1786506606003976 us; error 54 us; skew 500 ppm
Aug 12 03:50:06 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:06.010732 28229 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:50:06.011260 28229 webserver.cc:460] Webserver started at http://127.7.132.130:33379/ using document root <none> and password file <none>
I20260812 03:50:06.011554 28229 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:06.011638 28229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:50:06 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:06.035866 28229 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260812 03:50:06.037067 28259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:06.037226 28229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:06.037313 28229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "594ad7fdfb1446848366b404620d4f98"
format_stamp: "Formatted at 2026-08-12 03:50:05 on dist-test-slave-39l8"
server_key: "c26550785eb1234bb754f66982f58199"
server_key_iv: "2737928e0fb3fb2762c4d7a4c1c9918c"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:06.037777 28229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:06.060520 28229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:06.063198 28229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:06.063476 28229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:06.063753 28229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:06.064163 28229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:06.064272 28229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:06.064405 28229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:06.064464 28229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:06.071548 28229 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:40787
I20260812 03:50:06.072130 28229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:50:06.074589 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 28229
I20260812 03:50:06.074931 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.132.148:56579/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:41145
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:33207
--webserver_port=43995
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:56579/kms with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:06.075241 28430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:40787 every 8 connection(s)
Aug 12 03:50:06 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:06.101991 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:06.083877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:40445 (local address 127.7.132.190:33207)
0812 03:50:06.083930 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:06.083932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:06.087236 (+ 3304us) server_negotiation.cc:407] Connection header received
0812 03:50:06.089876 (+ 2640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:06.089878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:06.089899 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:06.089934 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:06.090903 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.091679 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.093306 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.093607 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.093657 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:06.097796 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:06.097798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:06.097798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:06.097807 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:06.099639 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:06.100263 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:06.100264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:06.100264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:06.100328 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:06.100632 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:06.100633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:06.100633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:06.100779 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:06.100837 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:06.101776 (+ 939us) server_negotiation.cc:299] Negotiation successful
0812 03:50:06.101889 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 03:50:06.102857 28439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:06.083518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33207 (local address 127.7.132.130:40445)
0812 03:50:06.087055 (+ 3537us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:06.087114 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:06.087862 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:06.090193 (+ 2331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:06.090199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:06.090461 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:06.090777 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:06.090784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.091850 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.091851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:06.093185 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:06.093199 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.093781 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.093782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:06.093920 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:06.096198 (+ 2278us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:06.096209 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:06.097667 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:06.099762 (+ 2095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:06.099769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:06.099771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:06.100146 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:06.100404 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:06.100404 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:06.100405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:06.100541 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:06.100875 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:06.100881 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:06.101045 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 03:50:06.102395 (+ 1350us) client_negotiation.cc:240] Negotiation successful
0812 03:50:06.102579 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3485,"thread_start_us":125,"threads_started":1}
I20260812 03:50:06.103646 28431 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33207
I20260812 03:50:06.103744 28431 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:06.103929 28431 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
W20260812 03:50:06.104645 28431 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33207 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:50:06.104733 28431 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:06.104794 28431 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:06.105165 28431 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:06.105242 28431 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
W20260812 03:50:06.105585 28431 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:50:06.197506 28435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:06.197815 28435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:06.197875 28435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:06.199501 28435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:06.199576 28435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:06.199615 28435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:50:06.201620 28435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.132.148:56579/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:41145
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=43995
--tserver_master_addrs=127.7.132.190:33207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:06.202057 28435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:06.202309 28435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:06.209246 28445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:06.209553 28450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:06.210903 28435 server_base.cc:1034] running on GCE node
W20260812 03:50:06.218434 28447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:06.218819 28435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:06.219108 28435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:06.220257 28435 hybrid_clock.cc:648] HybridClock initialized: now 1786506606220229 us; error 45 us; skew 500 ppm
Aug 12 03:50:06 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:06.223927 28435 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:50:06.224429 28435 webserver.cc:460] Webserver started at http://127.7.132.131:43995/ using document root <none> and password file <none>
I20260812 03:50:06.224709 28435 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:06.224784 28435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:50:06 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:06.257845 28435 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260812 03:50:06.259029 28463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:06.259204 28435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:06.259294 28435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "f4aec3e0d0bc4733bf7c887400c01d01"
format_stamp: "Formatted at 2026-08-12 03:50:05 on dist-test-slave-39l8"
server_key: "eeefc9db1b352d024267b9bd72b2e161"
server_key_iv: "f4314584585d81b94c5b9ab0f9c910d5"
server_key_version: "kuduclusterkey@0"
I20260812 03:50:06.259759 28435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:06.296525 28435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:06.298866 28435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:06.299014 28435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:06.299261 28435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:06.299685 28435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:06.299742 28435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:06.299777 28435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:06.299841 28435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:06.307737 28435 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:41145
I20260812 03:50:06.307807 28635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:41145 every 8 connection(s)
I20260812 03:50:06.308228 28435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:50:06.314083 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 28435
Aug 12 03:50:06 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:06.319423 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:06.309511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41911 (local address 127.7.132.190:33207)
0812 03:50:06.309560 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:06.309563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:06.309993 (+ 430us) server_negotiation.cc:407] Connection header received
0812 03:50:06.310678 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:06.310680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:06.310694 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:06.310764 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:06.311478 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.311983 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.313468 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.313744 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.313790 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:06.316239 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:06.316241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:06.316241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:06.316249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:06.317803 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:06.318324 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:06.318326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:06.318327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:06.318396 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:06.318699 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:06.318700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:06.318701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:06.318870 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:06.318934 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:06.319251 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 03:50:06.319329 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:50:06.320082 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:06.309195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33207 (local address 127.7.132.131:41911)
0812 03:50:06.309851 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:06.309898 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:06.310560 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:06.310856 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:06.310860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:06.311093 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:06.311363 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:06.311370 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.312121 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.312123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:06.313364 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:06.313367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.313901 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.313902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:06.314027 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:06.314551 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:06.314563 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:06.316144 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:06.317924 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:06.317931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:06.317933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:06.318230 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:06.318478 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:06.318479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:06.318480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:06.318624 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:06.318963 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:06.318970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:06.319108 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 03:50:06.319228 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 03:50:06.319888 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":115,"threads_started":1}
I20260812 03:50:06.320884 28637 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33207
I20260812 03:50:06.320991 28637 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:06.321187 28637 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
W20260812 03:50:06.321945 28637 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33207 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:50:06.322026 28637 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:06.322091 28637 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:06.322467 28637 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:06.322569 28637 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
W20260812 03:50:06.323084 28637 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:50:06.323406 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:06.314623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.7.132.190:33207)
0812 03:50:06.315217 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:06.315218 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:06.315225 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:50:06.315243 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:06.315243 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:06.315252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:06.315336 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:06.316257 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.317029 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.318149 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:06.318422 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:06.318502 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:06.320107 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:06.320108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:06.320109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:06.320116 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:06.321902 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:06.322349 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:06.322350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:06.322350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:06.322440 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:06.322740 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:06.322742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:06.322742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:06.322877 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:06.323056 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:06.323245 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 03:50:06.323321 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":72,"threads_started":1}
Aug 12 03:50:06 dist-test-slave-39l8 krb5kdc[26392](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:06.882133 28223 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:06.882295 28223 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:06.882743 27920 ts_manager.cc:194] Registered new tserver with Master: 43c9d19d20e9497faaf9f98ea5cb4e7e (127.7.132.129:33965)
I20260812 03:50:06.883740 27920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:51445
I20260812 03:50:07.105777 28431 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:07.105947 28431 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:07.106315 27920 ts_manager.cc:194] Registered new tserver with Master: 594ad7fdfb1446848366b404620d4f98 (127.7.132.130:40787)
I20260812 03:50:07.106894 27920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:40445
Aug 12 03:50:07 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
I20260812 03:50:07.323269 28637 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:07.323439 28637 heartbeater.cc:511] Master 127.7.132.190:33207 requested a full tablet report, sending...
I20260812 03:50:07.323771 27920 ts_manager.cc:194] Registered new tserver with Master: f4aec3e0d0bc4733bf7c887400c01d01 (127.7.132.131:41145)
I20260812 03:50:07.324244 27920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:41911
I20260812 03:50:07.325414 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:50:07.332989 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.326479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.7.132.190:33207)
0812 03:50:07.326590 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:07.326592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:07.326767 (+ 175us) server_negotiation.cc:407] Connection header received
0812 03:50:07.326852 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:07.326853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:07.326867 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:07.326933 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:07.327489 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.328070 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.328744 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.328956 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.329030 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.330254 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:07.330256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:07.330256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:07.330265 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:07.331758 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:07.332141 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:07.332143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:07.332143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:07.332212 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:07.332426 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:07.332427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:07.332428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:07.332535 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:07.332639 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:07.332785 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 03:50:07.332869 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
I20260812 03:50:07.334962 27920 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:50:07.336661 27920 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:50:07.343868 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.339837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.7.132.131:41145)
0812 03:50:07.340343 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:07.340348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:07.340359 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:50:07.340391 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:07.340393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:07.340408 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:07.340637 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:07.340777 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.341441 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.342735 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.343437 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.343520 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.343582 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:07.343752 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 03:50:07.343788 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":59,"threads_started":1}
I20260812 03:50:07.343866 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.339413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:41145 (local address 127.0.0.1:55484)
0812 03:50:07.339770 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:07.339963 (+ 193us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:07.340199 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:07.340639 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:07.340642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:07.340644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:07.340715 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.340717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.341530 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.341531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.342632 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.342634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.343535 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.343536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.343616 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.343629 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:50:07.343732 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 03:50:07.343786 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":89,"threads_started":1}
I20260812 03:50:07.344930 28514 tablet_service.cc:1467] Processing CreateTablet for tablet 60f0bace1ed440d395abfb412c8f8e6c (DEFAULT_TABLE table=test-table [id=691f5d5ad2d649ceaca2f928f056c41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:07.345305 28514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f0bace1ed440d395abfb412c8f8e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:07.347186 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.339542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.7.132.129:33965)
0812 03:50:07.339701 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:07.339706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:07.344097 (+ 4391us) server_negotiation.cc:407] Connection header received
0812 03:50:07.344218 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:07.344221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:07.344240 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:07.344389 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:07.344628 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.345444 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.346242 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.346810 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.346890 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.346946 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:07.347081 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 03:50:07.347123 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260812 03:50:07.347440 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.339416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:33965 (local address 127.0.0.1:55518)
0812 03:50:07.344044 (+ 4628us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:07.344053 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:07.344124 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:07.344507 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:07.344508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:07.344510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:07.344541 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.344543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.345533 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.345534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.346146 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.346156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.346901 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.346902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.346959 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.346963 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:50:07.347321 (+ 358us) client_negotiation.cc:240] Negotiation successful
0812 03:50:07.347366 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4616,"thread_start_us":83,"threads_started":1}
I20260812 03:50:07.348153 28124 tablet_service.cc:1467] Processing CreateTablet for tablet 60f0bace1ed440d395abfb412c8f8e6c (DEFAULT_TABLE table=test-table [id=691f5d5ad2d649ceaca2f928f056c41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:07.348487 28124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f0bace1ed440d395abfb412c8f8e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:07.352799 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.339635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:40787 (local address 127.0.0.1:41902)
0812 03:50:07.347836 (+ 8201us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:07.347846 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:07.347885 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:07.348968 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:07.348969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:07.348971 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:07.349001 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.349002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.350424 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.350425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.351560 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.351565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.352560 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.352561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.352621 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.352624 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:50:07.352698 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:50:07.352735 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8189,"thread_start_us":81,"threads_started":1}
I20260812 03:50:07.352823 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.339931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.7.132.130:40787)
0812 03:50:07.348720 (+ 8789us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:07.348724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:07.348746 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:50:07.348779 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:07.348781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:07.348797 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:07.348881 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:07.349090 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.350317 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.351671 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.352458 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.352556 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.352611 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:07.352712 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:50:07.352749 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8754,"thread_start_us":67,"threads_started":1}
I20260812 03:50:07.353578 28304 tablet_service.cc:1467] Processing CreateTablet for tablet 60f0bace1ed440d395abfb412c8f8e6c (DEFAULT_TABLE table=test-table [id=691f5d5ad2d649ceaca2f928f056c41e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:07.353878 28304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60f0bace1ed440d395abfb412c8f8e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:07.354871 28686 tablet_bootstrap.cc:492] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e: Bootstrap starting.
I20260812 03:50:07.355664 28686 tablet_bootstrap.cc:654] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:07.356174 28686 log.cc:824] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:07.357383 28691 tablet_bootstrap.cc:492] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Bootstrap starting.
I20260812 03:50:07.358109 28691 tablet_bootstrap.cc:654] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:07.358516 28691 log.cc:824] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:07.358844 28685 tablet_bootstrap.cc:492] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Bootstrap starting.
I20260812 03:50:07.359289 28691 tablet_bootstrap.cc:492] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: No bootstrap required, opened a new log
I20260812 03:50:07.359371 28691 ts_tablet_manager.cc:1397] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 03:50:07.359762 28685 tablet_bootstrap.cc:654] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:07.360217 28685 log.cc:824] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:07.361088 28685 tablet_bootstrap.cc:492] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: No bootstrap required, opened a new log
I20260812 03:50:07.361157 28685 ts_tablet_manager.cc:1397] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:50:07.362129 28691 raft_consensus.cc:348] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.362284 28691 raft_consensus.cc:374] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:07.362346 28691 raft_consensus.cc:729] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 594ad7fdfb1446848366b404620d4f98, State: Initialized, Role: FOLLOWER
I20260812 03:50:07.362469 28691 consensus_queue.cc:260] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.362778 28691 ts_tablet_manager.cc:1428] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 03:50:07.362795 28431 heartbeater.cc:503] Master 127.7.132.190:33207 was elected leader, sending a full tablet report...
I20260812 03:50:07.364220 28685 raft_consensus.cc:348] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.364368 28685 raft_consensus.cc:374] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:07.364399 28685 raft_consensus.cc:729] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4aec3e0d0bc4733bf7c887400c01d01, State: Initialized, Role: FOLLOWER
I20260812 03:50:07.364499 28685 consensus_queue.cc:260] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.364812 28685 ts_tablet_manager.cc:1428] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 03:50:07.364899 28637 heartbeater.cc:503] Master 127.7.132.190:33207 was elected leader, sending a full tablet report...
I20260812 03:50:07.367298 28686 tablet_bootstrap.cc:492] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e: No bootstrap required, opened a new log
I20260812 03:50:07.367379 28686 ts_tablet_manager.cc:1397] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260812 03:50:07.369658 28686 raft_consensus.cc:348] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.369799 28686 raft_consensus.cc:374] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:07.369844 28686 raft_consensus.cc:729] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c9d19d20e9497faaf9f98ea5cb4e7e, State: Initialized, Role: FOLLOWER
I20260812 03:50:07.369957 28686 consensus_queue.cc:260] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.370209 28686 ts_tablet_manager.cc:1428] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:50:07.370249 28223 heartbeater.cc:503] Master 127.7.132.190:33207 was elected leader, sending a full tablet report...
Aug 12 03:50:07 dist-test-slave-39l8 krb5kdc[26392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506606, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for HTTP/127.7.132.148@KRBTEST.COM
W20260812 03:50:07.559854 28638 tablet.cc:2369] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:50:07.584977 28432 tablet.cc:2369] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:50:07.604856 28224 tablet.cc:2369] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:50:07.688005 28698 raft_consensus.cc:482] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:50:07.688151 28698 raft_consensus.cc:504] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.688531 28698 leader_election.cc:288] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43c9d19d20e9497faaf9f98ea5cb4e7e (127.7.132.129:33965), 594ad7fdfb1446848366b404620d4f98 (127.7.132.130:40787)
I20260812 03:50:07.693718 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.689206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41667 (local address 127.7.132.130:40787)
0812 03:50:07.689256 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:07.689258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:07.689705 (+ 447us) server_negotiation.cc:407] Connection header received
0812 03:50:07.689809 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:07.689811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:07.689821 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:07.689849 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:07.690154 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.690954 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.692664 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.693284 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.693337 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.693346 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:07.693598 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 03:50:07.693641 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:50:07.694274 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.688891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:40787 (local address 127.7.132.131:41667)
0812 03:50:07.689636 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:07.689647 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:07.689737 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:07.689922 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:07.689923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:07.689925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:07.690059 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.690062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.691062 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.691063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.692506 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.692520 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.693427 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.693429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.693505 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.693509 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:50:07.694037 (+ 528us) client_negotiation.cc:240] Negotiation successful
0812 03:50:07.694083 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":108,"threads_started":1}
I20260812 03:50:07.694911 28363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f0bace1ed440d395abfb412c8f8e6c" candidate_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "594ad7fdfb1446848366b404620d4f98" is_pre_election: true
I20260812 03:50:07.695135 28363 raft_consensus.cc:2454] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4aec3e0d0bc4733bf7c887400c01d01 in term 0.
I20260812 03:50:07.696072 28466 leader_election.cc:302] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 594ad7fdfb1446848366b404620d4f98, f4aec3e0d0bc4733bf7c887400c01d01; no voters:
I20260812 03:50:07.696241 28698 raft_consensus.cc:2781] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:50:07.696290 28698 raft_consensus.cc:482] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:50:07.696311 28698 raft_consensus.cc:3037] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:07.697458 28698 raft_consensus.cc:504] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.697616 28698 leader_election.cc:288] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [CANDIDATE]: Term 1 election: Requested vote from peers 43c9d19d20e9497faaf9f98ea5cb4e7e (127.7.132.129:33965), 594ad7fdfb1446848366b404620d4f98 (127.7.132.130:40787)
I20260812 03:50:07.697974 28363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f0bace1ed440d395abfb412c8f8e6c" candidate_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "594ad7fdfb1446848366b404620d4f98"
I20260812 03:50:07.698062 28363 raft_consensus.cc:3037] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:07.698983 28363 raft_consensus.cc:2454] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4aec3e0d0bc4733bf7c887400c01d01 in term 1.
I20260812 03:50:07.699219 28466 leader_election.cc:302] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 594ad7fdfb1446848366b404620d4f98, f4aec3e0d0bc4733bf7c887400c01d01; no voters:
I20260812 03:50:07.699334 28698 raft_consensus.cc:2781] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:07.699484 28698 raft_consensus.cc:686] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 1 LEADER]: Becoming Leader. State: Replica: f4aec3e0d0bc4733bf7c887400c01d01, State: Running, Role: LEADER
I20260812 03:50:07.699591 28698 consensus_queue.cc:237] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } }
I20260812 03:50:07.700459 27920 catalog_manager.cc:5495] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4aec3e0d0bc4733bf7c887400c01d01 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 } health_report { overall_health: UNKNOWN } } }
I20260812 03:50:07.703768 28680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.689555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:38665 (local address 127.7.132.129:33965)
0812 03:50:07.689601 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:07.689603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:07.694760 (+ 5157us) server_negotiation.cc:407] Connection header received
0812 03:50:07.694895 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:07.694905 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:07.694916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:07.694948 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:07.695130 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.695830 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.702776 (+ 6946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.703371 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.703421 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.703431 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:07.703650 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 03:50:07.703695 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:50:07.704114 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.689348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:33965 (local address 127.7.132.131:38665)
0812 03:50:07.694680 (+ 5332us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:07.694692 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:07.694803 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:07.695019 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:07.695020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:07.695023 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:07.695055 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.695056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.701565 (+ 6509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.701566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.702644 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:07.702648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.703490 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.703491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:07.703569 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.703572 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:50:07.704005 (+ 433us) client_negotiation.cc:240] Negotiation successful
0812 03:50:07.704049 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5318,"thread_start_us":84,"threads_started":1}
I20260812 03:50:07.704701 28163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f0bace1ed440d395abfb412c8f8e6c" candidate_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" is_pre_election: true
I20260812 03:50:07.704901 28163 raft_consensus.cc:2454] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4aec3e0d0bc4733bf7c887400c01d01 in term 0.
I20260812 03:50:07.705030 28160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60f0bace1ed440d395abfb412c8f8e6c" candidate_uuid: "f4aec3e0d0bc4733bf7c887400c01d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e"
I20260812 03:50:07.705094 28160 raft_consensus.cc:3037] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:07.706103 28160 raft_consensus.cc:2454] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4aec3e0d0bc4733bf7c887400c01d01 in term 1.
I20260812 03:50:07.784276 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:07.780969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55500 (local address 127.7.132.131:41145)
0812 03:50:07.781026 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:07.781028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:07.781038 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:50:07.781090 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:07.781090 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:07.781112 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:07.781221 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:50:07.781375 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.782121 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.783296 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:07.783597 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:07.783675 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:07.783874 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:50:07.784008 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:50:07.784095 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:07.784146 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:50:07.784170 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:50:07.786377 28363 raft_consensus.cc:1261] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 1 FOLLOWER]: Refusing update from remote peer f4aec3e0d0bc4733bf7c887400c01d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:07.786394 28160 raft_consensus.cc:1261] T 60f0bace1ed440d395abfb412c8f8e6c P 43c9d19d20e9497faaf9f98ea5cb4e7e [term 1 FOLLOWER]: Refusing update from remote peer f4aec3e0d0bc4733bf7c887400c01d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:07.786760 28706 consensus_queue.cc:1035] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43c9d19d20e9497faaf9f98ea5cb4e7e" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:07.786870 28698 consensus_queue.cc:1035] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "594ad7fdfb1446848366b404620d4f98" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 40787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:07.790750 28717 mvcc.cc:205] Tried to move back new op lower bound from 7317531065490108416 to 7317531065138012160. Current Snapshot: MvccSnapshot[applied={T|T < 7317531065490108416}]
I20260812 03:50:07.791085 28722 mvcc.cc:205] Tried to move back new op lower bound from 7317531065490108416 to 7317531065138012160. Current Snapshot: MvccSnapshot[applied={T|T < 7317531065490108416}]
I20260812 03:50:07.791340 28719 mvcc.cc:205] Tried to move back new op lower bound from 7317531065490108416 to 7317531065138012160. Current Snapshot: MvccSnapshot[applied={T|T < 7317531065490108416}]
I20260812 03:50:07.793272 27921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:07.793358 27921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:07.794144 27921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0d79e01cf5f45b3b0585c3c56cb3018: Sending DeleteTablet for 3 replicas of tablet 60f0bace1ed440d395abfb412c8f8e6c
I20260812 03:50:07.794631 28514 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f0bace1ed440d395abfb412c8f8e6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:07 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55484
I20260812 03:50:07.795179 28124 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f0bace1ed440d395abfb412c8f8e6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:07 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:55518
I20260812 03:50:07.794929 28304 tablet_service.cc:1514] Processing DeleteTablet for tablet 60f0bace1ed440d395abfb412c8f8e6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:07 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:41902
I20260812 03:50:07.795528 28728 tablet_replica.cc:330] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: stopping tablet replica
I20260812 03:50:07.795670 28728 raft_consensus.cc:2229] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:50:07.795787 28728 raft_consensus.cc:2258] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:07.796065 28731 tablet_replica.cc:330] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: stopping tablet replica
I20260812 03:50:07.796262 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 27995
I20260812 03:50:07.796358 28731 raft_consensus.cc:2229] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:50:07.796430 28731 raft_consensus.cc:2258] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:07.802227 28728 ts_tablet_manager.cc:1905] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:07.804800 28728 ts_tablet_manager.cc:1918] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:07.804919 28728 log.cc:1196] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/60f0bace1ed440d395abfb412c8f8e6c
I20260812 03:50:07.805229 28728 ts_tablet_manager.cc:1939] T 60f0bace1ed440d395abfb412c8f8e6c P f4aec3e0d0bc4733bf7c887400c01d01: Deleting consensus metadata
I20260812 03:50:07.806351 28731 ts_tablet_manager.cc:1905] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:07.808401 28731 ts_tablet_manager.cc:1918] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:07.808531 28731 log.cc:1196] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/60f0bace1ed440d395abfb412c8f8e6c
I20260812 03:50:07.808857 28731 ts_tablet_manager.cc:1939] T 60f0bace1ed440d395abfb412c8f8e6c P 594ad7fdfb1446848366b404620d4f98: Deleting consensus metadata
I20260812 03:50:07.808897 27912 catalog_manager.cc:4841] TS f4aec3e0d0bc4733bf7c887400c01d01 (127.7.132.131:41145): tablet 60f0bace1ed440d395abfb412c8f8e6c (table test-table [id=691f5d5ad2d649ceaca2f928f056c41e]) successfully deleted
W20260812 03:50:07.809131 27914 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:33965 (error 108)
I20260812 03:50:07.809409 27912 catalog_manager.cc:4841] TS 594ad7fdfb1446848366b404620d4f98 (127.7.132.130:40787): tablet 60f0bace1ed440d395abfb412c8f8e6c (table test-table [id=691f5d5ad2d649ceaca2f928f056c41e]) successfully deleted
W20260812 03:50:07.809940 28681 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:50:07.809585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:33965 (local address 127.0.0.1:55532)
0812 03:50:07.809649 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:07.809662 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:07.809702 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:07.809885 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:33965: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 03:50:07.810055 27914 catalog_manager.cc:4584] TS 43c9d19d20e9497faaf9f98ea5cb4e7e (127.7.132.129:33965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60f0bace1ed440d395abfb412c8f8e6c: Network error: Client connection negotiation failed: client connection to 127.7.132.129:33965: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:50:07.810356 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 28229
I20260812 03:50:07.830886 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 28435
I20260812 03:50:07.842916 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 27882
I20260812 03:50:07.856609 7698 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:50:08.211762 7698 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:50:08.211851 7698 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:50:08.568039 7698 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:50:08.568 UTC [26429] LOG: received smart shutdown request
2026-08-12 03:50:08.572 UTC [26429] LOG: background worker "logical replication launcher" (PID 26448) exited with exit code 1
2026-08-12 03:50:08.572 UTC [26436] LOG: shutting down
2026-08-12 03:50:08.710 UTC [26429] LOG: database system is shut down
2026-08-12T03:50:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46195 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:50:08 dist-test-slave-39l8 krb5kdc[28742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:50:08 dist-test-slave-39l8 krb5kdc[28742](info): set up 2 sockets
Aug 12 03:50:08 dist-test-slave-39l8 krb5kdc[28742](info): commencing operation
krb5kdc: starting...
W20260812 03:50:10.027531 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:50:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:50:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:50:10.142417 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40173
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40173,127.7.132.189:45867,127.7.132.188:46445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:10.220507 28761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:10.220723 28761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:10.220767 28761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:10.221781 28761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:10.221843 28761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:10.221864 28761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:10.221892 28761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:10.221920 28761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:10.223443 28761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40173,127.7.132.189:45867,127.7.132.188:46445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:40173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:10.223698 28761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:10.223913 28761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:10.228405 28770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:10.228384 28773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:10.228384 28769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:10.228600 28761 server_base.cc:1034] running on GCE node
I20260812 03:50:10.228838 28761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:10.229102 28761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:10.230244 28761 hybrid_clock.cc:648] HybridClock initialized: now 1786506610230229 us; error 37 us; skew 500 ppm
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:10.232684 28761 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:10.233094 28761 webserver.cc:460] Webserver started at http://127.7.132.190:35015/ using document root <none> and password file <none>
I20260812 03:50:10.233316 28761 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:10.233374 28761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:10.233517 28761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:10.234488 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "744c7a5fed504792a8044f929d405f85"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "cb1b35ca302d42b70f110e183f0a717d"
server_key_iv: "ecc085d16d93f7a5bade5fca1a8722f6"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.234876 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "744c7a5fed504792a8044f929d405f85"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "cb1b35ca302d42b70f110e183f0a717d"
server_key_iv: "ecc085d16d93f7a5bade5fca1a8722f6"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.236200 28761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:10.236879 28788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:10.237059 28761 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:50:10.237151 28761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "744c7a5fed504792a8044f929d405f85"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "cb1b35ca302d42b70f110e183f0a717d"
server_key_iv: "ecc085d16d93f7a5bade5fca1a8722f6"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.237218 28761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:10.246877 28761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:10.248528 28761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:10.248683 28761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:10.252588 28761 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:40173
I20260812 03:50:10.252616 28865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:40173 every 8 connection(s)
I20260812 03:50:10.252955 28761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:10.253790 28866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:10.255370 28866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:50:10.257262 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 28761
I20260812 03:50:10.257385 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:50:10.257587 7698 external_mini_cluster.cc:1428] Setting key e1311fe01a07689d253b243215205b57
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:10.268011 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.256019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47542 (local address 127.7.132.190:40173)
0812 03:50:10.256202 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.256207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.256245 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:50:10.256829 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.256831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.256848 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.256923 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.257615 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.258818 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.260448 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.260966 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.261063 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.263917 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.263926 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.263926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.263934 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.266011 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.266497 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.266501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.266501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.266605 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.266897 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.266898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.266898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.267205 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.267323 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.267565 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.267793 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":153,"thread_start_us":86,"threads_started":1}
I20260812 03:50:10.268018 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.255979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40173 (local address 127.0.0.1:47542)
0812 03:50:10.256129 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.256178 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.256692 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.256948 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.256951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.257277 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.257507 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.257513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.258905 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.258907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.260339 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.260343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.261073 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.261074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.261219 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.261873 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.261883 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.263825 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.266107 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.266115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.266118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.266398 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.266689 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.266690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.266690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.266816 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.267315 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.267322 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.267400 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.267510 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.267716 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.7.132.189@KRBTEST.COM; defaulting to no policy
I20260812 03:50:10.269201 28866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:50:10.269524 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.269421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:34924)
0812 03:50:10.269469 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.269507 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:50:10.269644 28789 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
I20260812 03:50:10.270073 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.269981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:34926)
0812 03:50:10.270028 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.270050 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:50:10.270202 28866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:45867: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
I20260812 03:50:10.285349 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:45867
--webserver_interface=127.7.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40173,127.7.132.189:45867,127.7.132.188:46445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:10.319414 28866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } attempt: 1
I20260812 03:50:10.319921 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.319759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:34940)
0812 03:50:10.319834 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.319881 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 03:50:10.320349 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.320249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:34942)
0812 03:50:10.320299 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.320327 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:50:10.320487 28866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:45867: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
W20260812 03:50:10.370513 28873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:10.370755 28873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:10.370823 28873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:10.371986 28873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:10.372040 28873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:10.372054 28873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:10.372067 28873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:10.372102 28873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:10.374056 28873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40173,127.7.132.189:45867,127.7.132.188:46445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:45867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:10.374437 28873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:10.374696 28873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:10.379863 28884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:10.379868 28886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:10.380132 28888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:10.380856 28873 server_base.cc:1034] running on GCE node
I20260812 03:50:10.381137 28873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:10.381435 28873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:10.382604 28873 hybrid_clock.cc:648] HybridClock initialized: now 1786506610382584 us; error 36 us; skew 500 ppm
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:10.385030 28873 init.cc:377] Logged in from keytab as kudu/127.7.132.189@KRBTEST.COM (short username kudu)
I20260812 03:50:10.385464 28873 webserver.cc:460] Webserver started at http://127.7.132.189:33763/ using document root <none> and password file <none>
I20260812 03:50:10.385708 28873 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:10.385777 28873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:10.385931 28873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:10.387027 28873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/data/instance:
uuid: "38eca07f64e34b6a9e7aaabfc256c07a"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "eefe7778cea1e8d09eb7611baee87eda"
server_key_iv: "6e15d64d3bfc8e06a2d40275be5424e2"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.387377 28873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/wal/instance:
uuid: "38eca07f64e34b6a9e7aaabfc256c07a"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "eefe7778cea1e8d09eb7611baee87eda"
server_key_iv: "6e15d64d3bfc8e06a2d40275be5424e2"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.388644 28866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } attempt: 2
I20260812 03:50:10.388927 28873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:50:10.389031 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.388896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:34956)
0812 03:50:10.388955 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.388987 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:50:10.389386 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.389295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:34972)
0812 03:50:10.389341 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.389365 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:50:10.389515 28866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:45867: Network error: Client connection negotiation failed: client connection to 127.7.132.189:45867: connect: Connection refused (error 111)
I20260812 03:50:10.389755 28895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:10.389923 28873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:10.390008 28873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/wal
uuid: "38eca07f64e34b6a9e7aaabfc256c07a"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "eefe7778cea1e8d09eb7611baee87eda"
server_key_iv: "6e15d64d3bfc8e06a2d40275be5424e2"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.390089 28873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:10.401834 28873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:10.403733 28873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:10.403889 28873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:10.407888 28873 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:45867
I20260812 03:50:10.407932 28972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:45867 every 8 connection(s)
I20260812 03:50:10.408366 28873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
I20260812 03:50:10.408955 28975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:10.410399 28975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:10.418095 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 28873
I20260812 03:50:10.418118 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.411132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.7.132.190:40173)
0812 03:50:10.411202 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.411205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.411428 (+ 223us) server_negotiation.cc:407] Connection header received
0812 03:50:10.411833 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.411834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.411844 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.411893 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.412500 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.412965 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.413839 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.414018 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.414069 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.415873 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.415875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.415875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.415881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.417008 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.417344 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.417345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.417345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.417393 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.417587 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.417588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.417588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.417676 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.417721 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.417948 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.418027 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:50:10.418133 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.411108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40173 (local address 127.0.0.1:47546)
0812 03:50:10.411321 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.411354 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.411760 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.411964 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.411967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.412248 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.412428 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.412433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.413042 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.413042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.413768 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.413770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.414088 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.414089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.414182 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.414556 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.414564 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.415797 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.417071 (+ 1274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.417078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.417079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.417281 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.417447 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.417447 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.417448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.417536 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.417727 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.417731 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.417827 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.417899 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.417997 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260812 03:50:10.418207 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-1/wal/instance
I20260812 03:50:10.418381 7698 external_mini_cluster.cc:1428] Setting key c4d45d52e48bc2fab49d4b3184c254f0
I20260812 03:50:10.419009 28975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:50:10.426402 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.419229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:34978)
0812 03:50:10.419279 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.419289 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.419332 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.419617 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.419618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.419722 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.419763 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.419764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.420700 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.420701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.421403 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.421406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.422005 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.422005 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.422058 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.422463 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.422467 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.423666 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.425516 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.425517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.425517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.425663 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.425876 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.425876 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.425876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.425921 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.426204 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.426205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.426236 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.426283 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.426350 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:50:10.426507 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.419308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.7.132.189:45867)
0812 03:50:10.419473 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.419477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.419500 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:50:10.419522 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.419524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.419537 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.419608 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.419831 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.420617 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.421514 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.421933 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.422002 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.423747 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.423755 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.423756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.423763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.425434 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.425742 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.425744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.425745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.425807 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.425974 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.425975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.425975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.426135 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.426219 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.426315 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.426415 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260812 03:50:10.427063 28975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:50:10.427387 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.427285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41532)
0812 03:50:10.427319 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.427365 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.7.132.188@KRBTEST.COM; defaulting to no policy
W20260812 03:50:10.427536 28898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
I20260812 03:50:10.427984 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.427884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41542)
0812 03:50:10.427942 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.427968 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:50:10.428104 28975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:46445: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
I20260812 03:50:10.443486 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:46445
--webserver_interface=127.7.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:40173,127.7.132.189:45867,127.7.132.188:46445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:10.477336 28975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } attempt: 1
I20260812 03:50:10.477866 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.477734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41550)
0812 03:50:10.477799 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.477838 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:50:10.478209 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.478107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41552)
0812 03:50:10.478152 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.478175 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 03:50:10.478295 28975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:46445: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
I20260812 03:50:10.480775 28866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } attempt: 3
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:50:10.487773 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.481067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:34994)
0812 03:50:10.481122 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.481134 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.481180 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.481358 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.481359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.481519 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.481587 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.481588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.482256 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.482258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.482830 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.482832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.483201 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.483202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.483292 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.483782 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.483786 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.485146 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.486840 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.486841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.486841 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.487001 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.487221 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.487222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.487222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.487272 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.487533 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.487533 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.487571 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.487623 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.487693 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:50:10.487833 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.481165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.7.132.189:45867)
0812 03:50:10.481233 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.481235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.481246 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:50:10.481272 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.481273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.481282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.481353 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.481668 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.482151 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.482911 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.483127 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.483204 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.485233 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.485235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.485235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.485243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.486749 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.487073 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.487075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.487075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.487143 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.487336 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.487337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.487337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.487469 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.487536 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.487667 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.487753 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:50:10.488247 28866 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:50:10.488559 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.488468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41568)
0812 03:50:10.488511 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.488534 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 03:50:10.488873 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.488790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41570)
0812 03:50:10.488828 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.488861 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:50:10.488991 28866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:46445: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
I20260812 03:50:10.520222 28866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } attempt: 1
I20260812 03:50:10.520777 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.520604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41584)
0812 03:50:10.520687 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.520738 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 03:50:10.521132 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.521046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41586)
0812 03:50:10.521077 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.521107 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 03:50:10.521250 28866 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:46445: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
W20260812 03:50:10.526499 28987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:10.526741 28987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:10.526782 28987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:10.527876 28987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:10.527910 28987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:10.527935 28987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:10.527947 28987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:10.527958 28987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:10.529520 28987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:40173,127.7.132.189:45867,127.7.132.188:46445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:46445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:10.529721 28987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:10.529886 28987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:10.534611 28993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:10.534612 28997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:10.534636 28995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:10.534641 28987 server_base.cc:1034] running on GCE node
I20260812 03:50:10.535034 28987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:10.535338 28987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:10.536497 28987 hybrid_clock.cc:648] HybridClock initialized: now 1786506610536486 us; error 38 us; skew 500 ppm
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:10.538828 28987 init.cc:377] Logged in from keytab as kudu/127.7.132.188@KRBTEST.COM (short username kudu)
I20260812 03:50:10.539285 28987 webserver.cc:460] Webserver started at http://127.7.132.188:46829/ using document root <none> and password file <none>
I20260812 03:50:10.539502 28987 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:10.539542 28987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:10.539623 28987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:10.540859 28987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/data/instance:
uuid: "4d523d08e76e4b1588ff52859c89e3a0"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "db1b1e0284a038f34af15a104165f6ce"
server_key_iv: "4cc90f0eff5b2eb628192466a0456801"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.541217 28987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/wal/instance:
uuid: "4d523d08e76e4b1588ff52859c89e3a0"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "db1b1e0284a038f34af15a104165f6ce"
server_key_iv: "4cc90f0eff5b2eb628192466a0456801"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.542469 28987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:50:10.543218 29004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:10.543390 28987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:10.543480 28987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/wal
uuid: "4d523d08e76e4b1588ff52859c89e3a0"
format_stamp: "Formatted at 2026-08-12 03:50:10 on dist-test-slave-39l8"
server_key: "db1b1e0284a038f34af15a104165f6ce"
server_key_iv: "4cc90f0eff5b2eb628192466a0456801"
server_key_version: "encryptionkey@0"
I20260812 03:50:10.543555 28987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:10.546427 28975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } attempt: 2
I20260812 03:50:10.546875 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.546755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41590)
0812 03:50:10.546798 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.546835 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 03:50:10.547216 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.547125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41592)
0812 03:50:10.547169 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.547199 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:50:10.547307 28975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:46445: Network error: Client connection negotiation failed: client connection to 127.7.132.188:46445: connect: Connection refused (error 111)
I20260812 03:50:10.554633 28987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:10.556387 28987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:10.556548 28987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:10.560489 28987 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:46445
I20260812 03:50:10.560518 29098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:46445 every 8 connection(s)
I20260812 03:50:10.560860 28987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
I20260812 03:50:10.561753 29099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:10.563385 29099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:10.569008 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 28987
I20260812 03:50:10.569159 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786506457253237-7698-0/minicluster-data/master-2/wal/instance
I20260812 03:50:10.569337 7698 external_mini_cluster.cc:1428] Setting key f1313428ae8a12d960db703a6b4fdce4
I20260812 03:50:10.571863 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.564086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.7.132.190:40173)
0812 03:50:10.564153 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.564156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.564354 (+ 198us) server_negotiation.cc:407] Connection header received
0812 03:50:10.564827 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.564828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.564851 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.564899 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.565423 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.565868 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.566814 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.567001 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.567054 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.569064 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.569066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.569066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.569072 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.570570 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.570973 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.570974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.570974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.571031 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.571271 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.571272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.571273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.571377 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.571448 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.571697 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.571781 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:50:10.572017 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.564044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:40173 (local address 127.0.0.1:47554)
0812 03:50:10.564236 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.564272 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.564748 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.564960 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.564963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.565187 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.565357 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.565361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.565943 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.565943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.566697 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.566741 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.567073 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.567074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.567168 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.567607 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.567615 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.568958 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.570665 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.570672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.570674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.570898 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.571096 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.571097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.571097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.571203 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.571442 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.571448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.571577 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.571678 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.571813 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
I20260812 03:50:10.572618 29099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:50:10.578009 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.569720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.7.132.188:46445)
0812 03:50:10.569893 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.569897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.569920 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:50:10.569976 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.569978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.569992 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.570084 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.570657 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.571304 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.571999 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.572495 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.572583 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.575197 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.575221 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.575222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.575229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.576823 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.577171 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.577174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.577174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.577227 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.577441 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.577442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.577442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.577646 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.577798 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.577843 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.577912 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260812 03:50:10.579746 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.572896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:45867 (local address 127.0.0.1:35008)
0812 03:50:10.572934 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.572945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.572984 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.573128 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.573129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.573249 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.573290 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.573291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.574166 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.574167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.574932 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.574935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.575316 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.575317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.575396 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.575870 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.575874 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.576974 (+ 1100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.578766 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.578767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.578767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.578992 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.579202 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.579203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.579203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.579282 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.579494 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.579495 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.579542 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.579613 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.579678 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 03:50:10.579820 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.572955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.7.132.189:45867)
0812 03:50:10.572998 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.573000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.573008 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:50:10.573035 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.573038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.573051 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.573121 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.573383 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.574081 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.575006 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.575167 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.575312 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.577053 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.577054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.577055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.577060 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.578659 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.579088 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.579089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.579090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.579139 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.579344 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.579345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.579345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.579430 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.579475 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.579648 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.579733 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:50:10.580211 29099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:50:10.586057 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.580422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41614)
0812 03:50:10.580486 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.580494 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.580524 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.580629 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.580630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.580762 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.580791 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.580792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.581346 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.581347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.581789 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.581791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.582135 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.582136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.582180 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.582567 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.582570 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.583794 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.585119 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.585120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.585121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.585280 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.585465 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.585466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.585466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.585516 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.585777 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.585778 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.585813 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.585877 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.585963 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 03:50:10.586112 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.580449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.7.132.188:46445)
0812 03:50:10.580513 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.580514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.580521 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:50:10.580577 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.580577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.580585 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.580625 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.580854 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.581272 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.581868 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.582066 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.582136 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.583874 (+ 1738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.583875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.583876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.583882 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.585027 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.585349 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.585350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.585350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.585401 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.585586 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.585587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.585588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.585698 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.585781 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.585901 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.586008 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":48}
I20260812 03:50:10.586666 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.578822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.7.132.190:40173)
0812 03:50:10.578856 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.578857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.578865 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:50:10.578887 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.578888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.578896 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.578943 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.579552 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.580229 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.581116 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.581301 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.581369 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.583941 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.583942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.583942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.583949 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.585411 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.585786 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.585787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.585787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.585839 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.586043 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.586044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.586044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.586172 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.586348 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.586526 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.586609 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:50:10.587563 29099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0: Bootstrap starting.
I20260812 03:50:10.588526 29099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:10.588833 29099 log.cc:824] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:10.589617 29099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0: No bootstrap required, opened a new log
I20260812 03:50:10.591028 29099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
I20260812 03:50:10.591166 29099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:10.591187 29099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d523d08e76e4b1588ff52859c89e3a0, State: Initialized, Role: FOLLOWER
I20260812 03:50:10.591318 29099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:50:10.591636 29099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:10.591624 29108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.591728 29108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:50:10.593665 29123 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:50:10.593726 29123 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:50:10.595141 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.587559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.7.132.189:45867)
0812 03:50:10.587619 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.587621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.587681 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:50:10.587743 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.587744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.587753 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.587818 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.588216 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.588822 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.589616 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.589851 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.589918 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.592122 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.592124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.592124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.592132 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.593914 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.594311 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.594312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.594312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.594364 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.594558 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.594560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.594560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.594689 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.594944 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.594985 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.595060 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:50:10.596385 28866 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } attempt: 2
I20260812 03:50:10.597050 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:50:10.594493 28987 master_runner.cc:186] Error getting master registration for 127.7.132.190:40173: OK, instance_id { permanent_uuid: "744c7a5fed504792a8044f929d405f85" instance_seqno: 1786506610251448 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:50:10.597385 28987 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:50:10.604449 28868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.596601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41620)
0812 03:50:10.596640 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.596651 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.596687 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.596902 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.596903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.597060 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.597099 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.597101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.597960 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.597961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.598560 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.598562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.598993 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.598994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.599046 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.599406 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.599411 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.600884 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.603429 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.603430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.603431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.603636 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.603846 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.603847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.603847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.603905 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.604174 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.604175 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.604213 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.604267 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.604348 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 03:50:10.605046 29106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.596752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.7.132.188:46445)
0812 03:50:10.596791 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.596793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.596800 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:50:10.596817 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.596817 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.596825 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.597170 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.597198 (+ 28us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.597865 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.598637 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.598913 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.598978 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.600984 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.600985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.600985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.601011 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.603319 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.603714 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.603715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.603715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.603778 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.603965 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.603966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.603967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.604095 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.604149 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.604841 (+ 692us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.604925 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:50:10.607033 28866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85: Bootstrap starting.
I20260812 03:50:10.607465 28869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.598828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.7.132.190:40173)
0812 03:50:10.599031 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.599032 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.599039 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:50:10.599059 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.599060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.599069 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.599122 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.599812 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.600464 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.601777 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.602044 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.602086 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.604054 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.604055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.604055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.604061 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.605727 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.606411 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.606412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.606412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.606467 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.606954 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.606955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.606955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.607063 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.607126 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.607316 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.607388 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260812 03:50:10.607770 28866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:10.608088 28866 log.cc:824] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:10.608548 28981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.599216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.7.132.189:45867)
0812 03:50:10.599415 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.599417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.600416 (+ 999us) server_negotiation.cc:407] Connection header received
0812 03:50:10.600503 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.600504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.600514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.600542 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.601296 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.602014 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.602988 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.603306 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.603363 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.605788 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.605789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.605789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.605796 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.607343 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.607721 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.607723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.607723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.607776 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.607988 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.607989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.607989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.608103 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.608175 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.608402 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.608469 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260812 03:50:10.608875 28866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85: No bootstrap required, opened a new log
I20260812 03:50:10.611078 28866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
I20260812 03:50:10.611222 28866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:10.611270 28866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744c7a5fed504792a8044f929d405f85, State: Initialized, Role: FOLLOWER
I20260812 03:50:10.611423 28866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
I20260812 03:50:10.611744 28866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:10.611769 29139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.611867 29139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:50:10.614054 29157 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:50:10.614106 29157 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:50:10.614477 29133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.600065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41628 (local address 127.7.132.188:46445)
0812 03:50:10.600313 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.600315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.601393 (+ 1078us) server_negotiation.cc:407] Connection header received
0812 03:50:10.601476 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.601477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.601487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.601530 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.603816 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.604546 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.609368 (+ 4822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.609674 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.609715 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.611182 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.611183 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.611183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.611189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.613130 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.613538 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.613539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.613540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.613611 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.613842 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.613843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.613843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.613948 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.614042 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.614279 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.614355 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":78,"threads_started":1}
I20260812 03:50:10.614408 28761 master_runner.cc:186] Error getting master registration for 127.7.132.189:45867: OK, instance_id { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" instance_seqno: 1786506610406677 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:50:10.616827 28761 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:50:10.627553 29108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:50:10.627669 29108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
I20260812 03:50:10.627914 29108 leader_election.cc:288] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 744c7a5fed504792a8044f929d405f85 (127.7.132.190:40173), 38eca07f64e34b6a9e7aaabfc256c07a (127.7.132.189:45867)
I20260812 03:50:10.628188 28830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4d523d08e76e4b1588ff52859c89e3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744c7a5fed504792a8044f929d405f85" is_pre_election: true
I20260812 03:50:10.628364 28830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d523d08e76e4b1588ff52859c89e3a0 in term 0.
I20260812 03:50:10.628588 29007 leader_election.cc:302] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d523d08e76e4b1588ff52859c89e3a0, 744c7a5fed504792a8044f929d405f85; no voters:
I20260812 03:50:10.628733 29108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:50:10.628790 29108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:50:10.628816 29108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:10.630093 29108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
I20260812 03:50:10.630215 29108 leader_election.cc:288] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [CANDIDATE]: Term 1 election: Requested vote from peers 744c7a5fed504792a8044f929d405f85 (127.7.132.190:40173), 38eca07f64e34b6a9e7aaabfc256c07a (127.7.132.189:45867)
I20260812 03:50:10.630419 28830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4d523d08e76e4b1588ff52859c89e3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744c7a5fed504792a8044f929d405f85"
I20260812 03:50:10.630491 28830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:10.631428 28830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d523d08e76e4b1588ff52859c89e3a0 in term 1.
I20260812 03:50:10.631613 29007 leader_election.cc:302] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d523d08e76e4b1588ff52859c89e3a0, 744c7a5fed504792a8044f929d405f85; no voters:
I20260812 03:50:10.631945 29108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:10.632120 29108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [term 1 LEADER]: Becoming Leader. State: Replica: 4d523d08e76e4b1588ff52859c89e3a0, State: Running, Role: LEADER
I20260812 03:50:10.632247 29108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
I20260812 03:50:10.632634 29159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d523d08e76e4b1588ff52859c89e3a0. Latest consensus state: current_term: 1 leader_uuid: "4d523d08e76e4b1588ff52859c89e3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.632925 29159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:10.633306 29166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:10.633505 29166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:50:10.634991 28830 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [term 1 FOLLOWER]: Refusing update from remote peer 4d523d08e76e4b1588ff52859c89e3a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:10.635267 29159 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:10.636404 29139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d523d08e76e4b1588ff52859c89e3a0. Latest consensus state: current_term: 1 leader_uuid: "4d523d08e76e4b1588ff52859c89e3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.636475 29139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:50:10.637473 29159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d523d08e76e4b1588ff52859c89e3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.637552 29159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:10.638018 29167 mvcc.cc:205] Tried to move back new op lower bound from 7317531077158825984 to 7317531077150380032. Current Snapshot: MvccSnapshot[applied={T|T < 7317531077158825984}]
I20260812 03:50:10.638110 29139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d523d08e76e4b1588ff52859c89e3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.638167 29139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:50:10.638422 28975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } attempt: 3
I20260812 03:50:10.638751 29166 catalog_manager.cc:1347] Generated new cluster ID: f8501e1c60d64751b2409f92870e5871
I20260812 03:50:10.638801 29166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:50:10 dist-test-slave-39l8 krb5kdc[28742](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506610, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:50:10.645641 29168 mvcc.cc:205] Tried to move back new op lower bound from 7317531077158825984 to 7317531077150380032. Current Snapshot: MvccSnapshot[applied={T|T < 7317531077158825984}]
I20260812 03:50:10.646091 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.638680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:46445 (local address 127.0.0.1:41642)
0812 03:50:10.638754 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:10.638766 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:10.638810 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:10.638935 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:10.638935 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:10.639096 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:10.639138 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.639140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.640032 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.640033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.640769 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:10.640771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.641262 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.641263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:10.641325 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.641779 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:10.641783 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:10.643238 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:10.644955 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.644956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.644956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.645149 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.645376 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:10.645376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:10.645377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:10.645438 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:10.645748 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:10.645749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:10.645814 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 03:50:10.645882 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:50:10.646005 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 03:50:10.646864 29133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:10.638785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.7.132.188:46445)
0812 03:50:10.638820 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:10.638821 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:10.638831 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:50:10.638855 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:10.638856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:10.638866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:10.638921 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:10.639200 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.639947 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.640833 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:10.641162 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:10.641239 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:10.643321 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:10.643323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:10.643323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:10.643330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:10.644845 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.645235 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.645236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.645237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.645302 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:10.645505 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:10.645506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:10.645506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:10.645627 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:10.645688 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:10.646680 (+ 992us) server_negotiation.cc:299] Negotiation successful
0812 03:50:10.646787 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:50:10.648149 28975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: Bootstrap starting.
I20260812 03:50:10.648892 28975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:10.649200 28975 log.cc:824] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:10.650193 28975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: No bootstrap required, opened a new log
I20260812 03:50:10.652000 28975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
I20260812 03:50:10.652148 28975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:10.652191 28975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38eca07f64e34b6a9e7aaabfc256c07a, State: Initialized, Role: FOLLOWER
I20260812 03:50:10.652304 28975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } }
I20260812 03:50:10.652565 28975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:10.653856 28933 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4d523d08e76e4b1588ff52859c89e3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" is_pre_election: true
I20260812 03:50:10.654218 28932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4d523d08e76e4b1588ff52859c89e3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38eca07f64e34b6a9e7aaabfc256c07a"
I20260812 03:50:10.653784 28931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [term 0 FOLLOWER]: Advancing to term 1
W20260812 03:50:10.654438 28904 tablet.cc:2369] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:50:10.655833 28931 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [term 1 FOLLOWER]: Refusing update from remote peer 4d523d08e76e4b1588ff52859c89e3a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:10.656101 29108 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:10.658373 29195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d523d08e76e4b1588ff52859c89e3a0. Latest consensus state: current_term: 1 leader_uuid: "4d523d08e76e4b1588ff52859c89e3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.658435 29195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:50:10.658601 29199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4d523d08e76e4b1588ff52859c89e3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.658676 29199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:50:10.659354 29194 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:50:10.659416 29194 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:50:10.656276 29175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.659480 29175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:50:10.666100 29166 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:50:10.666754 29166 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:50:10.667376 29159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4d523d08e76e4b1588ff52859c89e3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "744c7a5fed504792a8044f929d405f85" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 40173 } } peers { permanent_uuid: "38eca07f64e34b6a9e7aaabfc256c07a" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 45867 } } peers { permanent_uuid: "4d523d08e76e4b1588ff52859c89e3a0" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 46445 } } }
I20260812 03:50:10.667464 29159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:10.698539 29166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d523d08e76e4b1588ff52859c89e3a0: Generated new TSK 0
I20260812 03:50:10.698781 29166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:50:11.614542 29157 catalog_manager.cc:1259] Loaded cluster ID: f8501e1c60d64751b2409f92870e5871
I20260812 03:50:11.614619 29157 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85: loading cluster ID for follower catalog manager: success
I20260812 03:50:11.615348 29157 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85: acquiring CA information for follower catalog manager: success
I20260812 03:50:11.615619 29157 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 744c7a5fed504792a8044f929d405f85: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:50:11.659888 29194 catalog_manager.cc:1259] Loaded cluster ID: f8501e1c60d64751b2409f92870e5871
I20260812 03:50:11.659962 29194 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: loading cluster ID for follower catalog manager: success
I20260812 03:50:11.660699 29194 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: acquiring CA information for follower catalog manager: success
I20260812 03:50:11.660948 29194 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 38eca07f64e34b6a9e7aaabfc256c07a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:50:11.761574 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 28761
I20260812 03:50:11.774767 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 28873
I20260812 03:50:11.784947 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 28987
2026-08-12T03:50:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2963 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:50:11 dist-test-slave-39l8 krb5kdc[29211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:50:11 dist-test-slave-39l8 krb5kdc[29211](info): set up 2 sockets
Aug 12 03:50:11 dist-test-slave-39l8 krb5kdc[29211](info): commencing operation
krb5kdc: starting...
W20260812 03:50:12.978404 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:50:13 dist-test-slave-39l8 krb5kdc[29211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:50:13.091538 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46551
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:13.170928 29236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:13.171144 29236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:13.171178 29236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:13.172318 29236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:13.172384 29236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:13.172406 29236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:13.172427 29236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:13.172458 29236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:13.173879 29236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:13.174291 29236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:13.174523 29236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:13.179131 29250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:13.179098 29248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:13.179490 29236 server_base.cc:1034] running on GCE node
W20260812 03:50:13.179121 29244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:13.179919 29236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:13.180200 29236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:13.181360 29236 hybrid_clock.cc:648] HybridClock initialized: now 1786506613181340 us; error 39 us; skew 500 ppm
Aug 12 03:50:13 dist-test-slave-39l8 krb5kdc[29211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506613, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:13.183748 29236 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:13.184198 29236 webserver.cc:460] Webserver started at http://127.7.132.190:39755/ using document root <none> and password file <none>
I20260812 03:50:13.184422 29236 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:13.184489 29236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:13.184633 29236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:13.185587 29236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "d5679de1fdfb456c90d54b60a6a42c99"
format_stamp: "Formatted at 2026-08-12 03:50:13 on dist-test-slave-39l8"
server_key: "73896d629c2b745e9852b994ef6ed78f"
server_key_iv: "0a0a4e3f32393fcee09c0b381a491eaf"
server_key_version: "encryptionkey@0"
I20260812 03:50:13.185937 29236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "d5679de1fdfb456c90d54b60a6a42c99"
format_stamp: "Formatted at 2026-08-12 03:50:13 on dist-test-slave-39l8"
server_key: "73896d629c2b745e9852b994ef6ed78f"
server_key_iv: "0a0a4e3f32393fcee09c0b381a491eaf"
server_key_version: "encryptionkey@0"
I20260812 03:50:13.187336 29236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 03:50:13.188026 29262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:13.188202 29236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:13.188321 29236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "d5679de1fdfb456c90d54b60a6a42c99"
format_stamp: "Formatted at 2026-08-12 03:50:13 on dist-test-slave-39l8"
server_key: "73896d629c2b745e9852b994ef6ed78f"
server_key_iv: "0a0a4e3f32393fcee09c0b381a491eaf"
server_key_version: "encryptionkey@0"
I20260812 03:50:13.188392 29236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:13.214178 29236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:13.215889 29236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:13.216049 29236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:13.220043 29236 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46551
I20260812 03:50:13.220079 29343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46551 every 8 connection(s)
I20260812 03:50:13.220432 29236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:13.221298 29345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:13.223708 29345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99: Bootstrap starting.
I20260812 03:50:13.224279 29345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:13.224565 29345 log.cc:824] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:13.225273 29345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99: No bootstrap required, opened a new log
I20260812 03:50:13.226506 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 29236
I20260812 03:50:13.226608 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:50:13.226831 7698 external_mini_cluster.cc:1428] Setting key 59a34748b6015e74b27893bec544fda5
I20260812 03:50:13.227293 29345 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5679de1fdfb456c90d54b60a6a42c99" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46551 } }
I20260812 03:50:13.227468 29345 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:13.227541 29345 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5679de1fdfb456c90d54b60a6a42c99, State: Initialized, Role: FOLLOWER
I20260812 03:50:13.227674 29345 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5679de1fdfb456c90d54b60a6a42c99" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46551 } }
I20260812 03:50:13.227749 29345 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:50:13.227771 29345 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:50:13.227821 29345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:13.228641 29345 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5679de1fdfb456c90d54b60a6a42c99" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46551 } }
I20260812 03:50:13.228768 29345 leader_election.cc:302] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5679de1fdfb456c90d54b60a6a42c99; no voters:
I20260812 03:50:13.228946 29345 leader_election.cc:288] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:50:13.228989 29352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:13.229107 29352 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [term 1 LEADER]: Becoming Leader. State: Replica: d5679de1fdfb456c90d54b60a6a42c99, State: Running, Role: LEADER
I20260812 03:50:13.229202 29345 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:13.229195 29352 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5679de1fdfb456c90d54b60a6a42c99" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46551 } }
I20260812 03:50:13.229921 29355 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5679de1fdfb456c90d54b60a6a42c99. Latest consensus state: current_term: 1 leader_uuid: "d5679de1fdfb456c90d54b60a6a42c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5679de1fdfb456c90d54b60a6a42c99" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46551 } } }
I20260812 03:50:13.229993 29355 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:13.230136 29357 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5679de1fdfb456c90d54b60a6a42c99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5679de1fdfb456c90d54b60a6a42c99" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46551 } } }
I20260812 03:50:13.230209 29357 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:13.230415 29370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:13.230962 29370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:50:13 dist-test-slave-39l8 krb5kdc[29211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:13.237313 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:13.227314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58122 (local address 127.7.132.190:46551)
0812 03:50:13.227486 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:13.227495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:13.227531 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:50:13.227648 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:13.227661 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:13.227710 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:13.227869 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:13.228488 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:13.229864 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:13.231001 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:13.231579 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:13.231704 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:13.234262 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:13.234265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:13.234266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:13.234275 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:13.235832 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:13.236228 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:13.236231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:13.236231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:13.236318 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:13.236530 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:13.236531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:13.236531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:13.236736 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:13.236848 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:13.236987 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 03:50:13.237081 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260812 03:50:13.237751 29370 catalog_manager.cc:1347] Generated new cluster ID: 95285c470e33417a89be462dee37165e
I20260812 03:50:13.237810 29370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:50:13.238368 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:50:13.247293 29370 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:50:13.248010 29370 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:50:13.260389 29370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5679de1fdfb456c90d54b60a6a42c99: Generated new TSK 0
I20260812 03:50:13.260555 29370 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:50:13.292403 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 29236
2026-08-12T03:50:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1514 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:50:13 dist-test-slave-39l8 krb5kdc[29392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:50:13 dist-test-slave-39l8 krb5kdc[29392](info): set up 2 sockets
Aug 12 03:50:13 dist-test-slave-39l8 krb5kdc[29392](info): commencing operation
krb5kdc: starting...
W20260812 03:50:14.493646 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:50:14 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:50:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:50:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:50:14.607911 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33511
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:14.686111 29410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:14.686342 29410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:14.686376 29410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:14.687532 29410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:14.687594 29410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:14.687618 29410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:14.687636 29410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:14.687668 29410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:14.689076 29410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:14.689357 29410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:14.689581 29410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:14.694321 29425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:14.694433 29410 server_base.cc:1034] running on GCE node
W20260812 03:50:14.694321 29423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:14.694581 29422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:14.694912 29410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:14.695262 29410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:14.696424 29410 hybrid_clock.cc:648] HybridClock initialized: now 1786506614696405 us; error 33 us; skew 500 ppm
Aug 12 03:50:14 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:14.698848 29410 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:14.699265 29410 webserver.cc:460] Webserver started at http://127.7.132.190:36477/ using document root <none> and password file <none>
I20260812 03:50:14.699491 29410 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:14.699558 29410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:14.699693 29410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:14.700749 29410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "365b80b5ef6746d086d217dcf4579741"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "b99d5fb83bbab632d6939574032f63b4"
server_key_iv: "706b6354b902e3b521bdac2741f30724"
server_key_version: "encryptionkey@0"
I20260812 03:50:14.701128 29410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "365b80b5ef6746d086d217dcf4579741"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "b99d5fb83bbab632d6939574032f63b4"
server_key_iv: "706b6354b902e3b521bdac2741f30724"
server_key_version: "encryptionkey@0"
I20260812 03:50:14.702466 29410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:50:14.703320 29444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:14.703507 29410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:14.703610 29410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "365b80b5ef6746d086d217dcf4579741"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "b99d5fb83bbab632d6939574032f63b4"
server_key_iv: "706b6354b902e3b521bdac2741f30724"
server_key_version: "encryptionkey@0"
I20260812 03:50:14.703706 29410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:14.721081 29410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:14.722867 29410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:14.723026 29410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:14.726889 29410 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:33511
I20260812 03:50:14.726922 29523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:33511 every 8 connection(s)
I20260812 03:50:14.727246 29410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:14.728160 29526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:14.730201 29526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Bootstrap starting.
I20260812 03:50:14.730681 29526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:14.730929 29526 log.cc:824] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:14.731597 29526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: No bootstrap required, opened a new log
I20260812 03:50:14.732923 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 29410
I20260812 03:50:14.733043 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:50:14.733253 7698 external_mini_cluster.cc:1428] Setting key 93b7759211909c18fcb9bf5e2905499e
I20260812 03:50:14.733510 29526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:14.733727 29526 raft_consensus.cc:374] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:14.733796 29526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365b80b5ef6746d086d217dcf4579741, State: Initialized, Role: FOLLOWER
I20260812 03:50:14.733989 29526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:14.734099 29526 raft_consensus.cc:388] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:50:14.734144 29526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:50:14.734192 29526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:14.735432 29526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:14.735553 29526 leader_election.cc:302] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 365b80b5ef6746d086d217dcf4579741; no voters:
I20260812 03:50:14.735761 29526 leader_election.cc:288] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:50:14.735781 29533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:14.735893 29533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 1 LEADER]: Becoming Leader. State: Replica: 365b80b5ef6746d086d217dcf4579741, State: Running, Role: LEADER
I20260812 03:50:14.736013 29526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:14.736021 29533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:14.736464 29535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365b80b5ef6746d086d217dcf4579741. Latest consensus state: current_term: 1 leader_uuid: "365b80b5ef6746d086d217dcf4579741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } } }
I20260812 03:50:14.736557 29535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:14.736766 29534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "365b80b5ef6746d086d217dcf4579741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } } }
I20260812 03:50:14.737077 29534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:14.737560 29543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:14.738106 29543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:50:14.739821 29543 catalog_manager.cc:1347] Generated new cluster ID: 4cde835cadb24d8ab242afe9a256a888
I20260812 03:50:14.739876 29543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:50:14 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:14.744411 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:14.733756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53870 (local address 127.7.132.190:33511)
0812 03:50:14.733942 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:14.733950 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:14.733998 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:50:14.734097 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:14.734102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:14.734177 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:14.734312 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:14.735204 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:14.736349 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:14.737419 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:14.737976 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:14.738098 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:14.740832 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:14.740836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:14.740836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:14.740848 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:14.742779 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:14.743170 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:14.743173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:14.743173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:14.743291 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:14.743504 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:14.743505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:14.743506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:14.743836 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:14.744020 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:14.744084 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 03:50:14.744217 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260812 03:50:14.766811 29543 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:50:14.767442 29543 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:50:14.769778 29543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Generated new TSK 0
I20260812 03:50:14.769939 29543 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:50:14.798982 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33511
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:14.878026 29563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:14.878250 29563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:14.878291 29563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:14.879424 29563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:14.879488 29563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:14.879527 29563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:50:14.880934 29563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:14.881201 29563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:14.881434 29563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:14.886217 29576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:14.886292 29563 server_base.cc:1034] running on GCE node
W20260812 03:50:14.886229 29580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:14.886238 29575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:14.886593 29563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:14.886838 29563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:14.887976 29563 hybrid_clock.cc:648] HybridClock initialized: now 1786506614887969 us; error 31 us; skew 500 ppm
Aug 12 03:50:14 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:14.890429 29563 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:50:14.890901 29563 webserver.cc:460] Webserver started at http://127.7.132.129:43755/ using document root <none> and password file <none>
I20260812 03:50:14.891163 29563 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:14.891244 29563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:14.891396 29563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:14.892374 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "aec81d6ccf204ed88c5bc182826cdec1"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "6cf24cec04427b8cba2ad562eb21dbaf"
server_key_iv: "f902968b6a6947962fd9b3d8ef0e7a48"
server_key_version: "encryptionkey@0"
I20260812 03:50:14.892742 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "aec81d6ccf204ed88c5bc182826cdec1"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "6cf24cec04427b8cba2ad562eb21dbaf"
server_key_iv: "f902968b6a6947962fd9b3d8ef0e7a48"
server_key_version: "encryptionkey@0"
I20260812 03:50:14.894178 29563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:50:14.894892 29591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:14.895066 29563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:50:14.895164 29563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "aec81d6ccf204ed88c5bc182826cdec1"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "6cf24cec04427b8cba2ad562eb21dbaf"
server_key_iv: "f902968b6a6947962fd9b3d8ef0e7a48"
server_key_version: "encryptionkey@0"
I20260812 03:50:14.895248 29563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:14.906453 29563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:14.908351 29563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:14.908509 29563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:14.908726 29563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:14.909026 29563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:14.909075 29563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:14.909108 29563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:14.909132 29563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:14.914841 29563 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45627
I20260812 03:50:14.914917 29790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45627 every 8 connection(s)
I20260812 03:50:14.915190 29563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:50:14 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506614, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:14.923381 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:14.915859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:41547 (local address 127.7.132.190:33511)
0812 03:50:14.915927 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:14.915930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:14.916110 (+ 180us) server_negotiation.cc:407] Connection header received
0812 03:50:14.916531 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:14.916532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:14.916542 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:14.916590 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:14.917402 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:14.917850 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:14.918828 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:14.919019 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:14.919071 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:14.921066 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:14.921067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:14.921067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:14.921072 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:14.922193 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:14.922529 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:14.922530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:14.922530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:14.922576 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:14.922880 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:14.922881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:14.922881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:14.922979 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:14.923033 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:14.923260 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 03:50:14.923321 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:50:14.923532 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:14.915844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:41547)
0812 03:50:14.915997 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:14.916030 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:14.916458 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:14.916637 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:14.916640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:14.917164 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:14.917334 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:14.917338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:14.917922 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:14.917923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:14.918665 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:14.918667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:14.919092 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:14.919093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:14.919197 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:14.919682 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:14.919691 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:14.920994 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:14.922259 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:14.922266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:14.922267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:14.922467 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:14.922667 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:14.922667 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:14.922668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:14.922821 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:14.923039 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:14.923044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:14.923150 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 03:50:14.923231 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:50:14.923345 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260812 03:50:14.923961 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 29563
I20260812 03:50:14.924094 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:50:14.924119 29794 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33511
I20260812 03:50:14.924229 29794 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:14.924366 7698 external_mini_cluster.cc:1428] Setting key 46d866c62e6851a69000ff48c10bf185
I20260812 03:50:14.924437 29794 heartbeater.cc:511] Master 127.7.132.190:33511 requested a full tablet report, sending...
I20260812 03:50:14.925024 29467 ts_manager.cc:194] Registered new tserver with Master: aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627)
I20260812 03:50:14.925630 29467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:41547
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:50:14.947515 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33511
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:15.027453 29801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:15.027665 29801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:15.027701 29801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:15.028805 29801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:15.028869 29801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:15.028910 29801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:50:15.030261 29801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:15.030499 29801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:15.030740 29801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:15.035466 29814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:15.035476 29810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:15.035627 29811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:15.035861 29801 server_base.cc:1034] running on GCE node
I20260812 03:50:15.036103 29801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:15.036346 29801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:15.037503 29801 hybrid_clock.cc:648] HybridClock initialized: now 1786506615037487 us; error 32 us; skew 500 ppm
Aug 12 03:50:15 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:15.042495 29801 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:50:15.042989 29801 webserver.cc:460] Webserver started at http://127.7.132.130:35751/ using document root <none> and password file <none>
I20260812 03:50:15.043176 29801 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:15.043217 29801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:15.043301 29801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:15.044317 29801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "652251f89c2a4ed388def07ff2d2ad26"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "8d9af2ee79bce10333b7af06c059ce88"
server_key_iv: "25ca55723269ab174a1176215ed243f2"
server_key_version: "encryptionkey@0"
I20260812 03:50:15.044613 29801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "652251f89c2a4ed388def07ff2d2ad26"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "8d9af2ee79bce10333b7af06c059ce88"
server_key_iv: "25ca55723269ab174a1176215ed243f2"
server_key_version: "encryptionkey@0"
I20260812 03:50:15.045820 29801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:50:15.046501 29824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.046674 29801 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:15.046787 29801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "652251f89c2a4ed388def07ff2d2ad26"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "8d9af2ee79bce10333b7af06c059ce88"
server_key_iv: "25ca55723269ab174a1176215ed243f2"
server_key_version: "encryptionkey@0"
I20260812 03:50:15.046844 29801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:15.065114 29801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:15.066815 29801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:15.066923 29801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:15.067085 29801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:15.067412 29801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:15.067479 29801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.067556 29801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:15.067589 29801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.073109 29801 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:44359
I20260812 03:50:15.073189 30009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:44359 every 8 connection(s)
I20260812 03:50:15.073484 29801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
Aug 12 03:50:15 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:15.081887 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.074219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:51755 (local address 127.7.132.190:33511)
0812 03:50:15.074289 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.074291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.074472 (+ 181us) server_negotiation.cc:407] Connection header received
0812 03:50:15.074934 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.074935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.074945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.075010 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:15.075855 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.076316 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.077206 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.077379 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.077429 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.079415 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:15.079416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:15.079416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:15.079423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:15.080820 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.081146 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.081147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.081147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.081195 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.081399 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.081400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.081400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.081492 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:15.081540 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.081764 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.081825 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:50:15.082050 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.074194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:51755)
0812 03:50:15.074358 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.074392 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.074857 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.075068 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.075071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.075604 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:15.075783 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.075787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.076391 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.076392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.077136 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.077138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.077450 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.077451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.077545 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.077954 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:15.077962 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:15.079331 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:15.080887 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:15.080892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:15.080894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:15.081085 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:15.081255 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:15.081256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:15.081256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:15.081346 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:15.081546 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:15.081549 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:15.081651 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.081732 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.081843 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260812 03:50:15.082660 30011 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33511
I20260812 03:50:15.082792 30011 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:15.082870 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 29801
I20260812 03:50:15.082994 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:50:15.082996 30011 heartbeater.cc:511] Master 127.7.132.190:33511 requested a full tablet report, sending...
I20260812 03:50:15.083220 7698 external_mini_cluster.cc:1428] Setting key a7b0d8c45396cb29199d852cea73e4a2
I20260812 03:50:15.083467 29467 ts_manager.cc:194] Registered new tserver with Master: 652251f89c2a4ed388def07ff2d2ad26 (127.7.132.130:44359)
I20260812 03:50:15.083950 29467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:51755
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:50:15.107492 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33511
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:15.188309 30022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:15.188481 30022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:15.188503 30022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:15.189528 30022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:15.189560 30022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:15.189585 30022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:50:15.190953 30022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:15.191133 30022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:15.191295 30022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:15.196148 30030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:15.196166 30031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:15.196211 30022 server_base.cc:1034] running on GCE node
W20260812 03:50:15.196113 30033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:15.196550 30022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:15.196759 30022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:15.197924 30022 hybrid_clock.cc:648] HybridClock initialized: now 1786506615197911 us; error 30 us; skew 500 ppm
Aug 12 03:50:15 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:15.200366 30022 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:50:15.200775 30022 webserver.cc:460] Webserver started at http://127.7.132.131:34965/ using document root <none> and password file <none>
I20260812 03:50:15.200989 30022 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:15.201054 30022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:15.201200 30022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:15.202188 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "bf13b7ffe2724dd78168c11ebafe0a0e"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "7a90a8d2c0f012a61627763d16714f95"
server_key_iv: "b81e0d6650a1552ebeec8e016170b27f"
server_key_version: "encryptionkey@0"
I20260812 03:50:15.202545 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "bf13b7ffe2724dd78168c11ebafe0a0e"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "7a90a8d2c0f012a61627763d16714f95"
server_key_iv: "b81e0d6650a1552ebeec8e016170b27f"
server_key_version: "encryptionkey@0"
I20260812 03:50:15.203923 30022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:15.204676 30042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.204891 30022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:50:15.204998 30022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "bf13b7ffe2724dd78168c11ebafe0a0e"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "7a90a8d2c0f012a61627763d16714f95"
server_key_iv: "b81e0d6650a1552ebeec8e016170b27f"
server_key_version: "encryptionkey@0"
I20260812 03:50:15.205081 30022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:15.225217 30022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:15.227046 30022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:15.227207 30022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:15.227438 30022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:15.227759 30022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:15.227810 30022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.227842 30022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:15.227866 30022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.233889 30022 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:44405
I20260812 03:50:15.233939 30224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:44405 every 8 connection(s)
I20260812 03:50:15.234270 30022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:50:15 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:15.242345 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 30022
I20260812 03:50:15.242462 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:50:15.242645 7698 external_mini_cluster.cc:1428] Setting key 50ba82f8eada388c3c0d5c173c5b65bf
I20260812 03:50:15.243203 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.235004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33767 (local address 127.7.132.190:33511)
0812 03:50:15.235047 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.235049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.235229 (+ 180us) server_negotiation.cc:407] Connection header received
0812 03:50:15.235856 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.235857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.235871 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.235920 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:15.236966 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.237469 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.238487 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.238675 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.238804 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.240717 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:15.240719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:15.240719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:15.240726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:15.241795 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.242155 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.242156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.242156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.242214 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.242472 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.242473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.242474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.242621 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:15.242762 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.243020 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.243104 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:50:15.243379 30228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.234964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:33767)
0812 03:50:15.235109 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.235149 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.235748 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.235984 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.235988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.236642 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:15.236880 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.236885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.237615 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.237616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.238385 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.238387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.238809 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.238811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.238935 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.239356 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:15.239363 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:15.240623 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:15.241872 (+ 1249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:15.241879 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:15.241881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:15.242075 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:15.242285 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:15.242286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:15.242286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:15.242388 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:15.242754 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:15.242761 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:15.242881 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.242994 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.243144 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260812 03:50:15.243939 30226 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33511
I20260812 03:50:15.244024 30226 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:15.244144 30226 heartbeater.cc:511] Master 127.7.132.190:33511 requested a full tablet report, sending...
I20260812 03:50:15.244511 29467 ts_manager.cc:194] Registered new tserver with Master: bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405)
I20260812 03:50:15.244810 29467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:33767
I20260812 03:50:15.254338 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:50:15.260950 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.255582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.7.132.190:33511)
0812 03:50:15.255620 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.255622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.255772 (+ 150us) server_negotiation.cc:407] Connection header received
0812 03:50:15.255847 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.255848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.255859 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.255925 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:15.256458 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.256903 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.257556 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.257723 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.257795 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.258808 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:15.258809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:15.258809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:15.258815 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:15.259860 (+ 1045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.260182 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.260183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.260183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.260240 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.260440 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.260440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.260441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.260574 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:15.260672 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.260776 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.260861 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:50:15.262424 29467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:50:15.264060 29467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:50:15.269946 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.266505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46294 (local address 127.7.132.129:45627)
0812 03:50:15.266659 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.266663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.266674 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:50:15.266738 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.266740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.266757 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.266843 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:15.267073 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.268159 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.269133 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.269576 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.269667 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.269723 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.269835 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.269871 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
I20260812 03:50:15.269970 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.266092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45627 (local address 127.0.0.1:46294)
0812 03:50:15.266550 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.266559 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.266596 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.266930 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.266931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.266933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:15.266983 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.266984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.268255 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.268256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.269049 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.269051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.269663 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.269664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.269737 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.269750 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.269822 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.269868 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":64,"threads_started":1}
I20260812 03:50:15.269994 30238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.266092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:44405 (local address 127.0.0.1:36664)
0812 03:50:15.266366 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.266376 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.266459 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.266761 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.266764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.266766 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:15.266821 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.266823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.267782 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.267783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.268906 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.268909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.269758 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.269760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.269826 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.269830 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.269873 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.269909 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260812 03:50:15.270124 30239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.266069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:44359 (local address 127.0.0.1:54758)
0812 03:50:15.266329 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.266365 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.266457 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.266944 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.266945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.266946 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:15.266983 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.266984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.268151 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.268151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.269083 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.269085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.269941 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.269942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.269993 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.269995 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.270040 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.270074 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":2,"thread_start_us":56,"threads_started":1}
I20260812 03:50:15.270200 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.266314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.7.132.131:44405)
0812 03:50:15.266498 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.266502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.266512 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:50:15.266548 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.266549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.266565 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.266641 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:15.266906 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.267674 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.268991 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.269617 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.269679 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.269725 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.270086 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.270121 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260812 03:50:15.270318 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.266169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54758 (local address 127.7.132.130:44359)
0812 03:50:15.266728 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.266732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.266741 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:50:15.266777 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.266780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.266794 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.266867 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:15.267072 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.268053 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.269166 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.269804 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.269865 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.269911 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.270203 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.270247 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":107,"threads_started":1}
I20260812 03:50:15.270967 30118 tablet_service.cc:1467] Processing CreateTablet for tablet 3f1247d58a274d61948522d17ef8298c (DEFAULT_TABLE table=test-table [id=3fd3a26048a943a3af2b5eccb28c1300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:15.271042 29909 tablet_service.cc:1467] Processing CreateTablet for tablet 3f1247d58a274d61948522d17ef8298c (DEFAULT_TABLE table=test-table [id=3fd3a26048a943a3af2b5eccb28c1300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:15.271210 29675 tablet_service.cc:1467] Processing CreateTablet for tablet 3f1247d58a274d61948522d17ef8298c (DEFAULT_TABLE table=test-table [id=3fd3a26048a943a3af2b5eccb28c1300]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:15.271330 30118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f1247d58a274d61948522d17ef8298c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:15.271394 29909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f1247d58a274d61948522d17ef8298c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:15.271508 29675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f1247d58a274d61948522d17ef8298c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:15.273774 30245 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Bootstrap starting.
I20260812 03:50:15.274353 30245 tablet_bootstrap.cc:654] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:15.274793 30249 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Bootstrap starting.
I20260812 03:50:15.275027 30248 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Bootstrap starting.
I20260812 03:50:15.275100 30245 log.cc:824] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:15.275708 30248 tablet_bootstrap.cc:654] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:15.275882 30245 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: No bootstrap required, opened a new log
I20260812 03:50:15.275961 30245 ts_tablet_manager.cc:1397] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:50:15.276254 30248 log.cc:824] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:15.277050 30249 tablet_bootstrap.cc:654] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:15.277611 30249 log.cc:824] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:15.278138 30248 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: No bootstrap required, opened a new log
I20260812 03:50:15.278211 30248 ts_tablet_manager.cc:1397] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:50:15.278146 30245 raft_consensus.cc:348] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.278348 30245 raft_consensus.cc:374] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:15.278383 30245 raft_consensus.cc:729] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf13b7ffe2724dd78168c11ebafe0a0e, State: Initialized, Role: FOLLOWER
I20260812 03:50:15.278488 30245 consensus_queue.cc:260] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.278779 30245 ts_tablet_manager.cc:1428] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:50:15.278817 30226 heartbeater.cc:503] Master 127.7.132.190:33511 was elected leader, sending a full tablet report...
I20260812 03:50:15.279258 30249 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: No bootstrap required, opened a new log
I20260812 03:50:15.279326 30249 ts_tablet_manager.cc:1397] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:50:15.280197 30248 raft_consensus.cc:348] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.280339 30248 raft_consensus.cc:374] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:15.280381 30248 raft_consensus.cc:729] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652251f89c2a4ed388def07ff2d2ad26, State: Initialized, Role: FOLLOWER
I20260812 03:50:15.280479 30248 consensus_queue.cc:260] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.280717 30248 ts_tablet_manager.cc:1428] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:50:15.280782 30011 heartbeater.cc:503] Master 127.7.132.190:33511 was elected leader, sending a full tablet report...
I20260812 03:50:15.281033 30249 raft_consensus.cc:348] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.281266 30249 raft_consensus.cc:374] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:15.281320 30249 raft_consensus.cc:729] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aec81d6ccf204ed88c5bc182826cdec1, State: Initialized, Role: FOLLOWER
I20260812 03:50:15.281455 30249 consensus_queue.cc:260] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.281746 29794 heartbeater.cc:503] Master 127.7.132.190:33511 was elected leader, sending a full tablet report...
I20260812 03:50:15.281751 30249 ts_tablet_manager.cc:1428] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260812 03:50:15.324004 30012 tablet.cc:2369] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:50:15.415875 29795 tablet.cc:2369] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:50:15.484903 30227 tablet.cc:2369] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:50:15.549525 30254 raft_consensus.cc:482] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:50:15.549676 30254 raft_consensus.cc:504] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.550076 30254 leader_election.cc:288] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 652251f89c2a4ed388def07ff2d2ad26 (127.7.132.130:44359), aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627)
I20260812 03:50:15.552989 30241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.550510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49685 (local address 127.7.132.130:44359)
0812 03:50:15.550572 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.550574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.550581 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:50:15.550611 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.550612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.550625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.550696 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:15.550877 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.551353 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.552265 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.552618 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.552721 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.552737 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.552876 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.552912 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:50:15.553006 30228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.550255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:44359 (local address 127.7.132.131:49685)
0812 03:50:15.550365 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.550379 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.550445 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.550723 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.550724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.550726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:15.550802 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.550803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.551454 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.551455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.552158 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.552160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.552717 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.552718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.552802 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.552805 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.552871 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.552912 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 03:50:15.553050 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.550440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:48005 (local address 127.7.132.129:45627)
0812 03:50:15.550491 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.550493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.550589 (+ 96us) server_negotiation.cc:407] Connection header received
0812 03:50:15.550646 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.550656 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.550671 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.550760 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:15.550924 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.551379 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.552306 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.552642 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.552758 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.552767 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.552924 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.552959 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:50:15.553107 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.550275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45627 (local address 127.7.132.131:48005)
0812 03:50:15.550502 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.550512 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.550590 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.550768 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.550769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.550771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:15.550853 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.550854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.551484 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.551485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.552216 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.552217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.552759 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.552760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.552838 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.552841 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.552905 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.552946 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":151,"threads_started":1}
I20260812 03:50:15.553453 29704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f1247d58a274d61948522d17ef8298c" candidate_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aec81d6ccf204ed88c5bc182826cdec1" is_pre_election: true
I20260812 03:50:15.553632 29704 raft_consensus.cc:2454] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf13b7ffe2724dd78168c11ebafe0a0e in term 0.
I20260812 03:50:15.553653 29934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f1247d58a274d61948522d17ef8298c" candidate_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652251f89c2a4ed388def07ff2d2ad26" is_pre_election: true
I20260812 03:50:15.553807 29934 raft_consensus.cc:2454] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf13b7ffe2724dd78168c11ebafe0a0e in term 0.
I20260812 03:50:15.553874 30052 leader_election.cc:302] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aec81d6ccf204ed88c5bc182826cdec1, bf13b7ffe2724dd78168c11ebafe0a0e; no voters:
I20260812 03:50:15.553990 30254 raft_consensus.cc:2781] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:50:15.554075 30254 raft_consensus.cc:482] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:50:15.554108 30254 raft_consensus.cc:3037] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:15.554926 30254 raft_consensus.cc:504] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.555090 30254 leader_election.cc:288] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [CANDIDATE]: Term 1 election: Requested vote from peers 652251f89c2a4ed388def07ff2d2ad26 (127.7.132.130:44359), aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627)
I20260812 03:50:15.555289 29704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f1247d58a274d61948522d17ef8298c" candidate_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aec81d6ccf204ed88c5bc182826cdec1"
I20260812 03:50:15.555312 29934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f1247d58a274d61948522d17ef8298c" candidate_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652251f89c2a4ed388def07ff2d2ad26"
I20260812 03:50:15.555377 29934 raft_consensus.cc:3037] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:15.555373 29704 raft_consensus.cc:3037] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:15.556244 29704 raft_consensus.cc:2454] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf13b7ffe2724dd78168c11ebafe0a0e in term 1.
I20260812 03:50:15.556238 29934 raft_consensus.cc:2454] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf13b7ffe2724dd78168c11ebafe0a0e in term 1.
I20260812 03:50:15.556422 30052 leader_election.cc:302] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aec81d6ccf204ed88c5bc182826cdec1, bf13b7ffe2724dd78168c11ebafe0a0e; no voters:
I20260812 03:50:15.556528 30254 raft_consensus.cc:2781] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:15.556654 30254 raft_consensus.cc:686] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 LEADER]: Becoming Leader. State: Replica: bf13b7ffe2724dd78168c11ebafe0a0e, State: Running, Role: LEADER
I20260812 03:50:15.556749 30254 consensus_queue.cc:237] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.557586 29469 catalog_manager.cc:5495] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e reported cstate change: term changed from 0 to 1, leader changed from <none> to bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131). New cstate: current_term: 1 leader_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } health_report { overall_health: HEALTHY } } }
I20260812 03:50:15.624711 30242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.622501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36666 (local address 127.7.132.131:44405)
0812 03:50:15.622570 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.622571 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.622579 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:50:15.622614 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.622618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.622635 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.622763 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:50:15.622844 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.623307 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.624010 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.624195 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.624243 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.624380 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:50:15.624477 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:50:15.624538 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.624566 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.624582 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:50:15.626467 29704 raft_consensus.cc:1261] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Refusing update from remote peer bf13b7ffe2724dd78168c11ebafe0a0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:15.626467 29934 raft_consensus.cc:1261] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Refusing update from remote peer bf13b7ffe2724dd78168c11ebafe0a0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:15.626863 30254 consensus_queue.cc:1035] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:15.626997 30261 consensus_queue.cc:1035] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:15.629297 30268 mvcc.cc:205] Tried to move back new op lower bound from 7317531097603383296 to 7317531097321017344. Current Snapshot: MvccSnapshot[applied={T|T < 7317531097603383296}]
I20260812 03:50:15.629426 30267 mvcc.cc:205] Tried to move back new op lower bound from 7317531097603383296 to 7317531097321017344. Current Snapshot: MvccSnapshot[applied={T|T < 7317531097603383296}]
I20260812 03:50:15.629868 30269 mvcc.cc:205] Tried to move back new op lower bound from 7317531097603383296 to 7317531097321017344. Current Snapshot: MvccSnapshot[applied={T|T < 7317531097603383296}]
I20260812 03:50:15.631887 29467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:15.631991 29467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:15.632783 29467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Sending DeleteTablet for 3 replicas of tablet 3f1247d58a274d61948522d17ef8298c
I20260812 03:50:15.633123 29675 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f1247d58a274d61948522d17ef8298c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:15 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:46294
I20260812 03:50:15.633154 29909 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f1247d58a274d61948522d17ef8298c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:15 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54758
I20260812 03:50:15.633369 30280 tablet_replica.cc:330] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: stopping tablet replica
I20260812 03:50:15.633435 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 29563
I20260812 03:50:15.633473 30280 raft_consensus.cc:2229] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:50:15.633584 30280 raft_consensus.cc:2258] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:15.639505 30280 ts_tablet_manager.cc:1905] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:15.641662 30280 ts_tablet_manager.cc:1918] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:15.641736 30280 log.cc:1196] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/3f1247d58a274d61948522d17ef8298c
I20260812 03:50:15.641979 30280 ts_tablet_manager.cc:1939] T 3f1247d58a274d61948522d17ef8298c P 652251f89c2a4ed388def07ff2d2ad26: Deleting consensus metadata
I20260812 03:50:15.642345 29447 catalog_manager.cc:4841] TS 652251f89c2a4ed388def07ff2d2ad26 (127.7.132.130:44359): tablet 3f1247d58a274d61948522d17ef8298c (table test-table [id=3fd3a26048a943a3af2b5eccb28c1300]) successfully deleted
I20260812 03:50:15.643570 30118 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f1247d58a274d61948522d17ef8298c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:15 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36664
I20260812 03:50:15.644522 30287 tablet_replica.cc:330] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: stopping tablet replica
I20260812 03:50:15.644647 30287 raft_consensus.cc:2229] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:50:15.644788 30287 raft_consensus.cc:2258] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:15.645287 30287 ts_tablet_manager.cc:1905] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:15.646790 30287 ts_tablet_manager.cc:1918] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:15.646862 30287 log.cc:1196] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/3f1247d58a274d61948522d17ef8298c
I20260812 03:50:15.647168 30287 ts_tablet_manager.cc:1939] T 3f1247d58a274d61948522d17ef8298c P bf13b7ffe2724dd78168c11ebafe0a0e: Deleting consensus metadata
I20260812 03:50:15.647598 29446 catalog_manager.cc:4841] TS bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405): tablet 3f1247d58a274d61948522d17ef8298c (table test-table [id=3fd3a26048a943a3af2b5eccb28c1300]) successfully deleted
W20260812 03:50:15.649837 29448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:45627 (error 108)
I20260812 03:50:15.650166 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 29801
I20260812 03:50:15.650344 30240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.650210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45627 (local address 127.0.0.1:46298)
0812 03:50:15.650270 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.650308 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 03:50:15.665591 29448 catalog_manager.cc:4584] TS aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f1247d58a274d61948522d17ef8298c: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45627: connect: Connection refused (error 111)
I20260812 03:50:15.672348 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 30022
I20260812 03:50:15.682405 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 29410
I20260812 03:50:15.692870 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33511
--webserver_interface=127.7.132.190
--webserver_port=36477
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:15.771045 30289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:15.771222 30289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:15.771243 30289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:15.772397 30289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:15.772439 30289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:15.772452 30289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:15.772464 30289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:15.772511 30289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:15.773957 30289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=36477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:15.774201 30289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:15.774423 30289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:15.784195 30304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:15.784221 30300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:15.784384 30289 server_base.cc:1034] running on GCE node
W20260812 03:50:15.784207 30310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:15.784781 30289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:15.785043 30289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:15.786221 30289 hybrid_clock.cc:648] HybridClock initialized: now 1786506615786208 us; error 39 us; skew 500 ppm
Aug 12 03:50:15 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:15.788590 30289 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:15.789001 30289 webserver.cc:460] Webserver started at http://127.7.132.190:36477/ using document root <none> and password file <none>
I20260812 03:50:15.789183 30289 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:15.789233 30289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:15.790484 30289 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:15.791157 30322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.791337 30289 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.791422 30289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "365b80b5ef6746d086d217dcf4579741"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "b99d5fb83bbab632d6939574032f63b4"
server_key_iv: "706b6354b902e3b521bdac2741f30724"
server_key_version: "encryptionkey@0"
I20260812 03:50:15.791805 30289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:15.810144 30289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:15.811951 30289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:15.812074 30289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:15.816048 30289 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:33511
I20260812 03:50:15.816115 30409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:33511 every 8 connection(s)
I20260812 03:50:15.816509 30289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:15.818137 30411 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:50:15.818362 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 30289
I20260812 03:50:15.818657 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:45627
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:33511
--webserver_port=43755
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:15.818974 30411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Bootstrap starting.
I20260812 03:50:15.828431 30411 log.cc:824] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:15.830183 30411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:15.830447 30411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Bootstrap complete.
I20260812 03:50:15.832803 30411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:15.833056 30411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365b80b5ef6746d086d217dcf4579741, State: Initialized, Role: FOLLOWER
I20260812 03:50:15.833295 30411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:15.833390 30411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:50:15.833452 30411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:50:15.833518 30411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:50:15.834506 30411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:15.834640 30411 leader_election.cc:302] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 365b80b5ef6746d086d217dcf4579741; no voters:
I20260812 03:50:15.834856 30411 leader_election.cc:288] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:50:15.834890 30421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:50:15.834996 30421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 2 LEADER]: Becoming Leader. State: Replica: 365b80b5ef6746d086d217dcf4579741, State: Running, Role: LEADER
I20260812 03:50:15.835093 30421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:15.835139 30411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:15.835356 30421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365b80b5ef6746d086d217dcf4579741. Latest consensus state: current_term: 2 leader_uuid: "365b80b5ef6746d086d217dcf4579741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } } }
I20260812 03:50:15.835429 30421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:15.835361 30422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "365b80b5ef6746d086d217dcf4579741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } } }
I20260812 03:50:15.835594 30422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:15.835623 30429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:15.836472 30429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:50:15.836604 30429 catalog_manager.cc:1259] Loaded cluster ID: 4cde835cadb24d8ab242afe9a256a888
I20260812 03:50:15.836686 30429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:50:15.837157 30429 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 03:50:15.839756 30429 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 03:50:15.841439 30429 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:50:15.841627 30429 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Loaded TSK: 0
I20260812 03:50:15.841840 30429 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:50:15.905539 30415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:15.905763 30415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:15.905804 30415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:15.906921 30415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:15.906976 30415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:15.907014 30415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:50:15.908434 30415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:45627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=43755
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:15.908687 30415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:15.908922 30415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:15.914460 30466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:15.914453 30461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:15.914461 30460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:15.914868 30415 server_base.cc:1034] running on GCE node
I20260812 03:50:15.915079 30415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:15.915308 30415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:15.916458 30415 hybrid_clock.cc:648] HybridClock initialized: now 1786506615916446 us; error 29 us; skew 500 ppm
Aug 12 03:50:15 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:15.918953 30415 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:50:15.919425 30415 webserver.cc:460] Webserver started at http://127.7.132.129:43755/ using document root <none> and password file <none>
I20260812 03:50:15.919652 30415 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:15.919731 30415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:15.921097 30415 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:15.921801 30477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:15.921962 30415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:15.922036 30415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "aec81d6ccf204ed88c5bc182826cdec1"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "6cf24cec04427b8cba2ad562eb21dbaf"
server_key_iv: "f902968b6a6947962fd9b3d8ef0e7a48"
server_key_version: "encryptionkey@0"
I20260812 03:50:15.922389 30415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:15.932128 30415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:15.934150 30415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:15.934293 30415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:15.934509 30415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:15.935006 30488 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:50:15.936444 30415 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:50:15.936501 30415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 03:50:15.936529 30415 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:50:15.937196 30415 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:50:15.937237 30415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:50:15.937281 30488 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Bootstrap starting.
I20260812 03:50:15.944804 30415 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45627
I20260812 03:50:15.944947 30664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45627 every 8 connection(s)
I20260812 03:50:15.945323 30415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:50:15.948931 30488 log.cc:824] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:15.951184 30488 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:15.951471 30488 tablet_bootstrap.cc:492] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Bootstrap complete.
I20260812 03:50:15.952562 30488 ts_tablet_manager.cc:1397] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260812 03:50:15.953816 30488 raft_consensus.cc:348] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.954077 30488 raft_consensus.cc:729] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aec81d6ccf204ed88c5bc182826cdec1, State: Initialized, Role: FOLLOWER
Aug 12 03:50:15 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:15.954222 30488 consensus_queue.cc:260] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } }
I20260812 03:50:15.954491 30488 ts_tablet_manager.cc:1428] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260812 03:50:15.954931 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 30415
I20260812 03:50:15.955252 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:44359
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:33511
--webserver_port=35751
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:15.958549 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.946020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:50009)
0812 03:50:15.946226 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.946269 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.946865 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.947708 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.947712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.948068 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:15.948325 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.948332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.949933 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.949934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.951183 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.951186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.951983 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.951984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.952147 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.952796 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:15.952805 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:15.954549 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:15.956781 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:15.956787 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:15.956789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:15.957027 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:15.957333 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:15.957334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:15.957335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:15.957455 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:15.957901 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:15.957906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:15.958045 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.958154 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.958318 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":122,"threads_started":1}
I20260812 03:50:15.958660 30670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.946190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:50009 (local address 127.7.132.190:33511)
0812 03:50:15.947175 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.947184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.947228 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:50:15.947378 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.947387 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.947461 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.947641 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:15.948472 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.949799 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.951285 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.951871 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.951982 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.954638 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:15.954642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:15.954642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:15.954653 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:15.956680 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.957111 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.957114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.957115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.957235 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.957527 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.957528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.957529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.957824 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:15.957917 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.958230 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.958399 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":918,"thread_start_us":70,"threads_started":1}
I20260812 03:50:15.959578 30666 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33511
I20260812 03:50:15.959712 30666 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:15.959896 30666 heartbeater.cc:511] Master 127.7.132.190:33511 requested a full tablet report, sending...
I20260812 03:50:15.960693 30348 ts_manager.cc:194] Registered new tserver with Master: aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627)
I20260812 03:50:15.961879 30348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:50009
I20260812 03:50:15.962905 30666 heartbeater.cc:503] Master 127.7.132.190:33511 was elected leader, sending a full tablet report...
Aug 12 03:50:15 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506615, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:50:15.970767 30682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.961514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.7.132.129:45627)
0812 03:50:15.961857 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:15.961861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:15.961869 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:50:15.961900 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:15.961902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:15.961917 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:15.962042 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:15.962680 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.963833 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.964931 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.965341 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.965413 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.967676 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:15.967695 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:15.967696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:15.967702 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:15.969427 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.969862 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.969865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.969866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.969926 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:15.970133 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:15.970133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:15.970134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:15.970285 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:15.970360 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:15.970546 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:50:15.970638 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":61,"threads_started":1}
I20260812 03:50:15.970768 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:15.961303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45627 (local address 127.0.0.1:46306)
0812 03:50:15.961661 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:15.961694 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:15.961753 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:15.962106 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:15.962112 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:15.962501 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:15.962581 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.962583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.963938 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.963939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.964829 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:15.964833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:15.965431 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:15.965432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:15.965521 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:15.965994 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:15.965999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:15.967568 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:15.969513 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:15.969519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:15.969521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:15.969784 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:15.970014 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:15.970015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:15.970016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:15.970075 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:15.970349 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:15.970356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:15.970401 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 03:50:15.970497 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:50:15.970606 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":78,"threads_started":1}
I20260812 03:50:15.971374 30532 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f1247d58a274d61948522d17ef8298c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:15 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:46306
I20260812 03:50:15.971963 30683 tablet_replica.cc:330] stopping tablet replica
I20260812 03:50:15.972091 30683 raft_consensus.cc:2229] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:50:15.972239 30683 raft_consensus.cc:2258] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:15.972553 30683 ts_tablet_manager.cc:1905] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:15.974745 30683 ts_tablet_manager.cc:1918] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:15.974825 30683 log.cc:1196] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/3f1247d58a274d61948522d17ef8298c
I20260812 03:50:15.975085 30683 ts_tablet_manager.cc:1939] T 3f1247d58a274d61948522d17ef8298c P aec81d6ccf204ed88c5bc182826cdec1: Deleting consensus metadata
I20260812 03:50:15.975610 30327 catalog_manager.cc:4841] TS aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627): tablet 3f1247d58a274d61948522d17ef8298c (table test-table [id=3fd3a26048a943a3af2b5eccb28c1300]) successfully deleted
W20260812 03:50:16.041554 30679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:16.041770 30679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:16.041810 30679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:16.042987 30679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:16.043045 30679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:16.043092 30679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:50:16.044528 30679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:44359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=35751
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:16.044782 30679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:16.044997 30679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:16.049742 30691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:16.049743 30690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:16.049880 30679 server_base.cc:1034] running on GCE node
W20260812 03:50:16.049765 30695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:16.050122 30679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:16.050349 30679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:16.051501 30679 hybrid_clock.cc:648] HybridClock initialized: now 1786506616051497 us; error 39 us; skew 500 ppm
Aug 12 03:50:16 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:16.053987 30679 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:50:16.054494 30679 webserver.cc:460] Webserver started at http://127.7.132.130:35751/ using document root <none> and password file <none>
I20260812 03:50:16.054757 30679 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:16.054849 30679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:16.056181 30679 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:16.056737 30705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.056891 30679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:16.056988 30679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "652251f89c2a4ed388def07ff2d2ad26"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "8d9af2ee79bce10333b7af06c059ce88"
server_key_iv: "25ca55723269ab174a1176215ed243f2"
server_key_version: "encryptionkey@0"
I20260812 03:50:16.057284 30679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:16.068764 30679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:16.071019 30679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:16.071171 30679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:16.071376 30679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:16.071668 30679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:16.071717 30679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.071758 30679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:16.071782 30679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.077646 30679 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:44359
I20260812 03:50:16.077710 30881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:44359 every 8 connection(s)
I20260812 03:50:16.078022 30679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:50:16.081465 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 30679
I20260812 03:50:16.081784 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:44405
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:33511
--webserver_port=34965
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:50:16 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:16.087414 30670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.078760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:57321 (local address 127.7.132.190:33511)
0812 03:50:16.078802 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.078804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.079009 (+ 205us) server_negotiation.cc:407] Connection header received
0812 03:50:16.079427 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.079428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.079438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.079487 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:16.080302 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.080774 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.081715 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.082026 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.082111 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.084307 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:16.084309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:16.084309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:16.084318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:16.085946 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:16.086336 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:16.086337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:16.086338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:16.086422 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:16.086676 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:16.086677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:16.086677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:16.086819 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:16.086896 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.087187 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.087305 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:50:16.087589 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.078751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:57321)
0812 03:50:16.078895 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.078927 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:16.079349 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:16.079534 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:16.079538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:16.080057 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:16.080229 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.080235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.080848 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.080849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.081637 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.081639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.082111 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.082112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.082208 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.082693 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:16.082788 (+ 95us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:16.084210 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:16.086042 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:16.086048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:16.086049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:16.086251 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:16.086494 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:16.086495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:16.086496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:16.086598 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:16.086888 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:16.086895 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:16.087019 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 03:50:16.087165 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 03:50:16.087377 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260812 03:50:16.088259 30883 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33511
I20260812 03:50:16.088368 30883 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:16.088538 30883 heartbeater.cc:511] Master 127.7.132.190:33511 requested a full tablet report, sending...
I20260812 03:50:16.088943 30348 ts_manager.cc:194] Registered new tserver with Master: 652251f89c2a4ed388def07ff2d2ad26 (127.7.132.130:44359)
I20260812 03:50:16.089316 30348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:57321
W20260812 03:50:16.163647 30888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:16.163858 30888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:16.163908 30888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:16.164979 30888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:16.165041 30888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:16.165083 30888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:50:16.166520 30888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:44405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=34965
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:16.166801 30888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:16.167028 30888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:16.171717 30899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:16.171790 30903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:16.171744 30900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:16.172019 30888 server_base.cc:1034] running on GCE node
I20260812 03:50:16.172248 30888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:16.172500 30888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:16.173652 30888 hybrid_clock.cc:648] HybridClock initialized: now 1786506616173636 us; error 33 us; skew 500 ppm
Aug 12 03:50:16 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:16.176070 30888 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:50:16.176507 30888 webserver.cc:460] Webserver started at http://127.7.132.131:34965/ using document root <none> and password file <none>
I20260812 03:50:16.176697 30888 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:16.176748 30888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:16.177956 30888 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:16.178507 30912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.178695 30888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:16.178819 30888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "bf13b7ffe2724dd78168c11ebafe0a0e"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "7a90a8d2c0f012a61627763d16714f95"
server_key_iv: "b81e0d6650a1552ebeec8e016170b27f"
server_key_version: "encryptionkey@0"
I20260812 03:50:16.179095 30888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:16.190828 30888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:16.192956 30888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:16.193109 30888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:16.193327 30888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:16.193626 30888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:16.193658 30888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.193707 30888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:16.193739 30888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.199599 30888 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:44405
I20260812 03:50:16.199672 31103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:44405 every 8 connection(s)
I20260812 03:50:16.199988 30888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:50:16 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:16.207724 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 30888
I20260812 03:50:16.208863 30670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.200654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33501 (local address 127.7.132.190:33511)
0812 03:50:16.200717 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.200719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.200943 (+ 224us) server_negotiation.cc:407] Connection header received
0812 03:50:16.201414 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.201415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.201427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.201479 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:16.202334 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.202909 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.203869 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.204080 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.204165 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.206234 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:16.206236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:16.206236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:16.206243 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:16.207433 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:16.207813 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:16.207814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:16.207815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:16.207873 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:16.208116 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:16.208117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:16.208117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:16.208297 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:16.208405 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.208683 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.208774 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 03:50:16.209010 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.200643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:33501)
0812 03:50:16.200801 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.200852 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:16.201310 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:16.201540 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:16.201545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:16.202057 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:16.202233 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.202237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.203000 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.203001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.203771 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.203774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.204173 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.204174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.204294 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.204860 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:16.204868 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:16.206141 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:16.207517 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:16.207523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:16.207526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:16.207736 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:16.207946 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:16.207947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:16.207947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:16.208043 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:16.208408 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:16.208415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:16.208539 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 03:50:16.208646 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 03:50:16.208795 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260812 03:50:16.209491 31106 heartbeater.cc:347] Connected to a master server at 127.7.132.190:33511
I20260812 03:50:16.209574 31106 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:16.209756 31106 heartbeater.cc:511] Master 127.7.132.190:33511 requested a full tablet report, sending...
I20260812 03:50:16.210132 30348 ts_manager.cc:194] Registered new tserver with Master: bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405)
I20260812 03:50:16.210486 30348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:33501
I20260812 03:50:16.214476 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.208116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.7.132.190:33511)
0812 03:50:16.208230 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.208232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.208394 (+ 162us) server_negotiation.cc:407] Connection header received
0812 03:50:16.208471 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.208473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.208484 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.208565 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:16.209244 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.209882 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.210654 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.210908 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.210976 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.212128 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:16.212129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:16.212129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:16.212136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:16.213320 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:16.213703 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:16.213704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:16.213705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:16.213761 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:16.213974 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:16.213975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:16.213976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:16.214083 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:16.214265 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.214302 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.214377 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":50,"threads_started":1}
I20260812 03:50:16.215200 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:50:16.217577 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.215537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53894 (local address 127.7.132.190:33511)
0812 03:50:16.215607 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.215608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.215672 (+ 64us) server_negotiation.cc:407] Connection header received
0812 03:50:16.215752 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.215753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.215778 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.215836 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:50:16.215908 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.216327 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.216985 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.217153 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.217199 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.217317 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:50:16.217388 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:50:16.217439 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.217478 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.217493 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:50:16.217873 30348 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:50:16.219538 30348 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:50:16.221712 30532 tablet_service.cc:1467] Processing CreateTablet for tablet f0474b5d4e174b41a8557d6a4883f00a (DEFAULT_TABLE table=test-table [id=099418edbf184408b18a7beaf84d6b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:16.221918 30532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0474b5d4e174b41a8557d6a4883f00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:16.223572 30488 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Bootstrap starting.
I20260812 03:50:16.223906 30488 tablet_bootstrap.cc:654] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:16.224449 30488 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: No bootstrap required, opened a new log
I20260812 03:50:16.224510 30488 ts_tablet_manager.cc:1397] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:50:16.224856 30488 raft_consensus.cc:348] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.224954 30488 raft_consensus.cc:374] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:16.225010 30488 raft_consensus.cc:729] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aec81d6ccf204ed88c5bc182826cdec1, State: Initialized, Role: FOLLOWER
I20260812 03:50:16.225064 30488 consensus_queue.cc:260] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.225167 30488 ts_tablet_manager.cc:1428] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:16.225829 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.221406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:44359 (local address 127.0.0.1:54772)
0812 03:50:16.221436 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.221447 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:16.221488 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:16.222049 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:16.222050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:16.222052 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:16.222145 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.222147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.223347 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.223348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.224457 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.224461 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.225557 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.225558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.225623 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.225644 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:50:16.225711 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:50:16.225760 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":73}
I20260812 03:50:16.226039 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.221493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:44405 (local address 127.0.0.1:36672)
0812 03:50:16.221618 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.221628 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:16.221668 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:16.222050 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:16.222051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:16.222053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:16.222151 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.222152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.223234 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.223236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.224933 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.224936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.225808 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.225809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.225877 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.225881 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:50:16.225933 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:50:16.225974 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":26,"thread_start_us":80,"threads_started":1}
I20260812 03:50:16.226089 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.221556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54772 (local address 127.7.132.130:44359)
0812 03:50:16.221859 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.221863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.221874 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:50:16.221913 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.221915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.221932 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.222088 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:16.222214 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.223245 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.224543 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.225468 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.225547 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.225605 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.225761 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.225831 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":97,"threads_started":1}
I20260812 03:50:16.226423 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.221670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.7.132.131:44405)
0812 03:50:16.221837 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.221842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.221851 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:50:16.221886 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.221888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.221902 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.221968 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:16.222231 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.223129 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.225027 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.225671 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.225729 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.225775 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.226337 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.226372 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260812 03:50:16.226802 30769 tablet_service.cc:1467] Processing CreateTablet for tablet f0474b5d4e174b41a8557d6a4883f00a (DEFAULT_TABLE table=test-table [id=099418edbf184408b18a7beaf84d6b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:16.226969 30999 tablet_service.cc:1467] Processing CreateTablet for tablet f0474b5d4e174b41a8557d6a4883f00a (DEFAULT_TABLE table=test-table [id=099418edbf184408b18a7beaf84d6b1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:16.227151 30769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0474b5d4e174b41a8557d6a4883f00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:16.227303 30999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0474b5d4e174b41a8557d6a4883f00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:16.229720 31118 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Bootstrap starting.
I20260812 03:50:16.230362 31118 tablet_bootstrap.cc:654] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:16.230897 31118 log.cc:824] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:16.231192 31117 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Bootstrap starting.
I20260812 03:50:16.231600 31118 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: No bootstrap required, opened a new log
I20260812 03:50:16.231657 31118 ts_tablet_manager.cc:1397] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:50:16.231775 31117 tablet_bootstrap.cc:654] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:16.232239 31117 log.cc:824] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:16.233031 31117 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: No bootstrap required, opened a new log
I20260812 03:50:16.233125 31117 ts_tablet_manager.cc:1397] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:50:16.233477 31118 raft_consensus.cc:348] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.233608 31118 raft_consensus.cc:374] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:16.233630 31118 raft_consensus.cc:729] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652251f89c2a4ed388def07ff2d2ad26, State: Initialized, Role: FOLLOWER
I20260812 03:50:16.233752 31118 consensus_queue.cc:260] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.234041 31118 ts_tablet_manager.cc:1428] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:50:16.234102 30883 heartbeater.cc:503] Master 127.7.132.190:33511 was elected leader, sending a full tablet report...
I20260812 03:50:16.234941 31117 raft_consensus.cc:348] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.235183 31117 raft_consensus.cc:374] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:16.235239 31117 raft_consensus.cc:729] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf13b7ffe2724dd78168c11ebafe0a0e, State: Initialized, Role: FOLLOWER
I20260812 03:50:16.235347 31117 consensus_queue.cc:260] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.235606 31117 ts_tablet_manager.cc:1428] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:50:16.235635 31106 heartbeater.cc:503] Master 127.7.132.190:33511 was elected leader, sending a full tablet report...
W20260812 03:50:16.328579 30884 tablet.cc:2369] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:50:16.447423 30667 tablet.cc:2369] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:50:16.450495 31107 tablet.cc:2369] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:50:16.551062 31121 raft_consensus.cc:482] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:50:16.551200 31121 raft_consensus.cc:504] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.551628 31121 leader_election.cc:288] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405), aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627)
I20260812 03:50:16.554527 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.551941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:57537 (local address 127.7.132.131:44405)
0812 03:50:16.551985 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.551987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.552143 (+ 156us) server_negotiation.cc:407] Connection header received
0812 03:50:16.552210 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.552211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.552227 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.552296 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:16.552444 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.552908 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.553825 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.554186 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.554280 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.554291 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.554419 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.554460 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:50:16.554581 31130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.551833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:44405 (local address 127.7.132.130:57537)
0812 03:50:16.552050 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.552058 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:16.552152 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:16.552300 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:16.552300 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:16.552302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:16.552375 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.552377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.552993 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.552994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.553718 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.553720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.554280 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.554281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.554348 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.554351 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:50:16.554414 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:50:16.554484 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":54,"thread_start_us":94,"threads_started":1}
I20260812 03:50:16.554739 31131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.551933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:42553 (local address 127.7.132.129:45627)
0812 03:50:16.552080 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.552081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.552088 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:50:16.552115 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.552115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.552137 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.552225 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:16.552404 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.552900 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.553807 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.554214 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.554297 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.554368 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.554498 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.554538 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":52,"threads_started":1}
I20260812 03:50:16.554598 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.551822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45627 (local address 127.7.132.130:42553)
0812 03:50:16.551887 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.551900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:16.551962 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:16.552242 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:16.552243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:16.552245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:16.552317 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.552319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.552991 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.552993 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.553713 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:16.553716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.554306 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.554307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:16.554389 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.554392 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:50:16.554480 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:50:16.554519 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:50:16.554913 31027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0474b5d4e174b41a8557d6a4883f00a" candidate_uuid: "652251f89c2a4ed388def07ff2d2ad26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" is_pre_election: true
I20260812 03:50:16.555058 31027 raft_consensus.cc:2454] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652251f89c2a4ed388def07ff2d2ad26 in term 0.
I20260812 03:50:16.555135 30586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0474b5d4e174b41a8557d6a4883f00a" candidate_uuid: "652251f89c2a4ed388def07ff2d2ad26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aec81d6ccf204ed88c5bc182826cdec1" is_pre_election: true
I20260812 03:50:16.555260 30586 raft_consensus.cc:2454] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 652251f89c2a4ed388def07ff2d2ad26 in term 0.
I20260812 03:50:16.555325 30707 leader_election.cc:302] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652251f89c2a4ed388def07ff2d2ad26, bf13b7ffe2724dd78168c11ebafe0a0e; no voters:
I20260812 03:50:16.555459 31121 raft_consensus.cc:2781] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:50:16.555517 31121 raft_consensus.cc:482] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:50:16.555547 31121 raft_consensus.cc:3037] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:16.556329 31121 raft_consensus.cc:504] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.556461 31121 leader_election.cc:288] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [CANDIDATE]: Term 1 election: Requested vote from peers bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405), aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627)
I20260812 03:50:16.556617 31027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0474b5d4e174b41a8557d6a4883f00a" candidate_uuid: "652251f89c2a4ed388def07ff2d2ad26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e"
I20260812 03:50:16.556689 31027 raft_consensus.cc:3037] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:16.557335 31027 raft_consensus.cc:2454] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652251f89c2a4ed388def07ff2d2ad26 in term 1.
I20260812 03:50:16.557446 30586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0474b5d4e174b41a8557d6a4883f00a" candidate_uuid: "652251f89c2a4ed388def07ff2d2ad26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aec81d6ccf204ed88c5bc182826cdec1"
I20260812 03:50:16.557502 30707 leader_election.cc:302] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 652251f89c2a4ed388def07ff2d2ad26, bf13b7ffe2724dd78168c11ebafe0a0e; no voters:
I20260812 03:50:16.557525 30586 raft_consensus.cc:3037] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:16.557601 31121 raft_consensus.cc:2781] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:16.557734 31121 raft_consensus.cc:686] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 LEADER]: Becoming Leader. State: Replica: 652251f89c2a4ed388def07ff2d2ad26, State: Running, Role: LEADER
I20260812 03:50:16.557840 31121 consensus_queue.cc:237] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.558275 30586 raft_consensus.cc:2454] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 652251f89c2a4ed388def07ff2d2ad26 in term 1.
I20260812 03:50:16.558669 30347 catalog_manager.cc:5495] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 652251f89c2a4ed388def07ff2d2ad26 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "652251f89c2a4ed388def07ff2d2ad26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } health_report { overall_health: UNKNOWN } } }
I20260812 03:50:16.579139 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.577194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.7.132.130:44359)
0812 03:50:16.577255 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:16.577257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:16.577264 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:50:16.577336 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:16.577337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:16.577350 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:16.577403 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:50:16.577506 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.577887 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.578504 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:16.578660 (+ 156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:16.578750 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:16.578833 (+ 83us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:50:16.578916 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:50:16.578967 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:16.579044 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:50:16.579061 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:50:16.580772 30586 raft_consensus.cc:1261] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Refusing update from remote peer 652251f89c2a4ed388def07ff2d2ad26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:16.580796 31027 raft_consensus.cc:1261] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 FOLLOWER]: Refusing update from remote peer 652251f89c2a4ed388def07ff2d2ad26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:16.581143 31132 consensus_queue.cc:1035] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:16.581278 31121 consensus_queue.cc:1035] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:16.583897 31138 mvcc.cc:205] Tried to move back new op lower bound from 7317531101512372224 to 7317531101421436928. Current Snapshot: MvccSnapshot[applied={T|T < 7317531101512372224}]
I20260812 03:50:16.584285 31139 mvcc.cc:205] Tried to move back new op lower bound from 7317531101512372224 to 7317531101421436928. Current Snapshot: MvccSnapshot[applied={T|T < 7317531101512372224}]
I20260812 03:50:16.585748 30348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:53894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:16.585845 30348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:53894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:16.586560 30348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Sending DeleteTablet for 3 replicas of tablet f0474b5d4e174b41a8557d6a4883f00a
I20260812 03:50:16.586862 30769 tablet_service.cc:1514] Processing DeleteTablet for tablet f0474b5d4e174b41a8557d6a4883f00a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:16 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54772
I20260812 03:50:16.586917 30532 tablet_service.cc:1514] Processing DeleteTablet for tablet f0474b5d4e174b41a8557d6a4883f00a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:16 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:46306
I20260812 03:50:16.587093 31152 tablet_replica.cc:330] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: stopping tablet replica
I20260812 03:50:16.587073 30999 tablet_service.cc:1514] Processing DeleteTablet for tablet f0474b5d4e174b41a8557d6a4883f00a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:16 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36672
I20260812 03:50:16.587178 31152 raft_consensus.cc:2229] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:50:16.587235 31152 raft_consensus.cc:2258] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:16.587287 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 30415
I20260812 03:50:16.594349 31153 tablet_replica.cc:330] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: stopping tablet replica
I20260812 03:50:16.594480 31153 raft_consensus.cc:2229] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:50:16.594586 31153 raft_consensus.cc:2258] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:16.594776 31149 tablet_replica.cc:330] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: stopping tablet replica
I20260812 03:50:16.594873 31149 raft_consensus.cc:2229] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:50:16.595057 31149 raft_consensus.cc:2258] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:16.595265 31153 ts_tablet_manager.cc:1905] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:16.597460 31153 ts_tablet_manager.cc:1918] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:16.597546 31153 log.cc:1196] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/f0474b5d4e174b41a8557d6a4883f00a
I20260812 03:50:16.597831 31153 ts_tablet_manager.cc:1939] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Deleting consensus metadata
I20260812 03:50:16.598253 30324 catalog_manager.cc:4841] TS bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405): tablet f0474b5d4e174b41a8557d6a4883f00a (table test-table [id=099418edbf184408b18a7beaf84d6b1a]) successfully deleted
W20260812 03:50:16.601473 30327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:45627 (error 108)
I20260812 03:50:16.601764 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 30679
I20260812 03:50:16.601778 31149 ts_tablet_manager.cc:1905] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:16.608001 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.607880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45627 (local address 127.0.0.1:46316)
0812 03:50:16.607929 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.607962 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:50:16.608101 30327 catalog_manager.cc:4584] TS aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f0474b5d4e174b41a8557d6a4883f00a: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45627: connect: Connection refused (error 111)
I20260812 03:50:16.613988 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.613824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:44359 (local address 127.0.0.1:54792)
0812 03:50:16.613901 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.613943 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.130:44359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:50:16.613960 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 30888
I20260812 03:50:16.622670 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 30289
I20260812 03:50:16.631598 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33511
--webserver_interface=127.7.132.190
--webserver_port=36477
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:16.710131 31155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:16.710343 31155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:16.710379 31155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:16.711560 31155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:16.711629 31155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:16.711653 31155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:16.711683 31155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:16.711705 31155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:16.713166 31155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=36477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:16.713445 31155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:16.713661 31155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:16.718374 31170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:16.718382 31164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:16.718386 31165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:16.718622 31155 server_base.cc:1034] running on GCE node
I20260812 03:50:16.718871 31155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:16.719153 31155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:16.720314 31155 hybrid_clock.cc:648] HybridClock initialized: now 1786506616720299 us; error 27 us; skew 500 ppm
Aug 12 03:50:16 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:16.722868 31155 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:16.723274 31155 webserver.cc:460] Webserver started at http://127.7.132.190:36477/ using document root <none> and password file <none>
I20260812 03:50:16.723451 31155 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:16.723491 31155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:16.724712 31155 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:16.725317 31181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.725483 31155 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:16.725589 31155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "365b80b5ef6746d086d217dcf4579741"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "b99d5fb83bbab632d6939574032f63b4"
server_key_iv: "706b6354b902e3b521bdac2741f30724"
server_key_version: "encryptionkey@0"
I20260812 03:50:16.725904 31155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:16.735706 31155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:16.737753 31155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:16.737897 31155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:16.741686 31155 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:33511
I20260812 03:50:16.741714 31265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:33511 every 8 connection(s)
I20260812 03:50:16.742050 31155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:16.743556 31267 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:50:16.744199 31267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Bootstrap starting.
I20260812 03:50:16.746592 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 31155
I20260812 03:50:16.746891 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:45627
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:33511
--webserver_port=43755
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:16.754367 31267 log.cc:824] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:16.756628 31267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:16.756907 31267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741: Bootstrap complete.
I20260812 03:50:16.758980 31267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:16.759195 31267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365b80b5ef6746d086d217dcf4579741, State: Initialized, Role: FOLLOWER
I20260812 03:50:16.759346 31267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:16.759403 31267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:50:16.759446 31267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:50:16.759490 31267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:50:16.760378 31267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:16.760484 31267 leader_election.cc:302] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 365b80b5ef6746d086d217dcf4579741; no voters:
I20260812 03:50:16.760675 31267 leader_election.cc:288] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:50:16.760732 31272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:50:16.760854 31272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [term 3 LEADER]: Becoming Leader. State: Replica: 365b80b5ef6746d086d217dcf4579741, State: Running, Role: LEADER
I20260812 03:50:16.760931 31267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:16.760950 31272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } }
I20260812 03:50:16.761150 31272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 365b80b5ef6746d086d217dcf4579741. Latest consensus state: current_term: 3 leader_uuid: "365b80b5ef6746d086d217dcf4579741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } } }
I20260812 03:50:16.761212 31272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:16.761152 31273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "365b80b5ef6746d086d217dcf4579741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "365b80b5ef6746d086d217dcf4579741" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33511 } } }
I20260812 03:50:16.761464 31273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 365b80b5ef6746d086d217dcf4579741 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:16.761395 31278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:16.762356 31278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:50:16.762509 31278 catalog_manager.cc:1259] Loaded cluster ID: 4cde835cadb24d8ab242afe9a256a888
I20260812 03:50:16.762658 31278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 03:50:16.762944 31278 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7faedccd29ad google::LogMessage::Fail() at ??:0
@ 0x7faedccd588a google::LogMessage::SendToLog() at ??:0
@ 0x7faedccd24de google::LogMessage::Flush() at ??:0
@ 0x7faedccd3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7faee4bf30f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7faee4bfc8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7faedeba00d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7faedeb95954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7faedf9f96db start_thread at ??:0
@ 0x7faedb7d061f clone at ??:0
W20260812 03:50:16.839967 31270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:16.840127 31270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:16.840147 31270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:16.841143 31270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:16.841181 31270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:16.841204 31270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:50:16.842541 31270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:45627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=43755
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:16.842803 31270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:16.843042 31270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:16.848799 31335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:16.848906 31270 server_base.cc:1034] running on GCE node
W20260812 03:50:16.848807 31340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:16.849059 31333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:16.849296 31270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:16.849534 31270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:16.850692 31270 hybrid_clock.cc:648] HybridClock initialized: now 1786506616850669 us; error 58 us; skew 500 ppm
Aug 12 03:50:16 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:16.853196 31270 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:50:16.853688 31270 webserver.cc:460] Webserver started at http://127.7.132.129:43755/ using document root <none> and password file <none>
I20260812 03:50:16.853927 31270 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:16.854004 31270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:16.855322 31270 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:16.856009 31349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.856196 31270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:16.856287 31270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "aec81d6ccf204ed88c5bc182826cdec1"
format_stamp: "Formatted at 2026-08-12 03:50:14 on dist-test-slave-39l8"
server_key: "6cf24cec04427b8cba2ad562eb21dbaf"
server_key_iv: "f902968b6a6947962fd9b3d8ef0e7a48"
server_key_version: "encryptionkey@0"
I20260812 03:50:16.856575 31270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:16.868640 31270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:16.870645 31270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:16.870854 31270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:16.871058 31270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:16.871572 31362 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:50:16.873483 31270 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:50:16.873543 31270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 03:50:16.873582 31270 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:50:16.874276 31270 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:50:16.874325 31270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:50:16.874389 31362 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Bootstrap starting.
I20260812 03:50:16.880414 31270 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45627
I20260812 03:50:16.880453 31539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45627 every 8 connection(s)
I20260812 03:50:16.880798 31270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:50:16.882907 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 31270
I20260812 03:50:16.883189 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:44359
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:33511
--webserver_port=35751
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:16.885113 31362 log.cc:824] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:16.887794 31362 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:16.888088 31362 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Bootstrap complete.
I20260812 03:50:16.889192 31362 ts_tablet_manager.cc:1397] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260812 03:50:16.890407 31362 raft_consensus.cc:348] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.890734 31362 raft_consensus.cc:729] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aec81d6ccf204ed88c5bc182826cdec1, State: Initialized, Role: FOLLOWER
I20260812 03:50:16.890887 31362 consensus_queue.cc:260] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:16.891131 31362 ts_tablet_manager.cc:1428] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
W20260812 03:50:16.936869 31544 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:50:16.881405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:59223)
0812 03:50:16.881592 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.881628 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:16.882055 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:16.936670 (+ 54615us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":121,"threads_started":1}
W20260812 03:50:16.937129 31540 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33511 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:33511: Client connection negotiation failed: client connection to 127.7.132.190:33511: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:50:16.937476 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.937359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:59577)
0812 03:50:16.937410 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.937453 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:50:16.937793 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:16.937703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:46511)
0812 03:50:16.937750 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:16.937773 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 03:50:16.937909 31540 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:50:16.965672 31546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:16.965842 31546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:16.965863 31546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:16.966974 31546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:16.967013 31546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:16.967037 31546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:50:16.968492 31546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:44359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=35751
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:16.968742 31546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:16.968976 31546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:16.973807 31559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:16.973814 31555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:16.973822 31557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:16.974225 31546 server_base.cc:1034] running on GCE node
I20260812 03:50:16.974442 31546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:16.974671 31546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:16.975855 31546 hybrid_clock.cc:648] HybridClock initialized: now 1786506616975834 us; error 37 us; skew 500 ppm
Aug 12 03:50:16 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:16.978324 31546 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:50:16.978775 31546 webserver.cc:460] Webserver started at http://127.7.132.130:35751/ using document root <none> and password file <none>
I20260812 03:50:16.979005 31546 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:16.979071 31546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:16.980286 31546 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 03:50:16.981010 31571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:16.981182 31546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:16.981276 31546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "652251f89c2a4ed388def07ff2d2ad26"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "8d9af2ee79bce10333b7af06c059ce88"
server_key_iv: "25ca55723269ab174a1176215ed243f2"
server_key_version: "encryptionkey@0"
I20260812 03:50:16.981596 31546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:16.993665 31546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:16.995442 31546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:16.995609 31546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:16.995823 31546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:16.996227 31580 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:50:16.997504 31546 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:50:16.997560 31546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260812 03:50:16.997598 31546 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:50:16.998185 31546 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:50:16.998237 31546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:50:16.998283 31580 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Bootstrap starting.
I20260812 03:50:16.999850 31580 log.cc:824] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:17.002696 31580 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:17.003005 31580 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Bootstrap complete.
I20260812 03:50:17.003180 31580 ts_tablet_manager.cc:1397] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260812 03:50:17.004585 31580 raft_consensus.cc:348] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:17.004882 31580 raft_consensus.cc:729] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652251f89c2a4ed388def07ff2d2ad26, State: Initialized, Role: FOLLOWER
I20260812 03:50:17.005040 31580 consensus_queue.cc:260] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:17.005142 31546 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:44359
I20260812 03:50:17.005333 31580 ts_tablet_manager.cc:1428] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260812 03:50:17.005496 31546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:50:17.005504 31749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:44359 every 8 connection(s)
I20260812 03:50:17.006498 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:17.006136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:55499)
0812 03:50:17.006299 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:17.006393 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
W20260812 03:50:17.006692 31752 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33511 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:33511: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
I20260812 03:50:17.006973 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:17.006891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:45665)
0812 03:50:17.006934 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:17.006958 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:50:17.007241 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:17.007171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:53557)
0812 03:50:17.007209 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:17.007228 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 03:50:17.007346 31752 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:50:17.013537 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 31546
I20260812 03:50:17.013787 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:44405
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:33511
--webserver_port=34965
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:41861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:17.095114 31756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:17.095288 31756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:17.095309 31756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:17.096330 31756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:17.096364 31756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:17.096387 31756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:50:17.097647 31756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:41861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:44405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=34965
--tserver_master_addrs=127.7.132.190:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:17.097801 31756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:17.097949 31756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:17.102535 31761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:17.102524 31775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:17.102751 31756 server_base.cc:1034] running on GCE node
W20260812 03:50:17.102538 31763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:17.103005 31756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:17.103255 31756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:17.104401 31756 hybrid_clock.cc:648] HybridClock initialized: now 1786506617104392 us; error 27 us; skew 500 ppm
Aug 12 03:50:17 dist-test-slave-39l8 krb5kdc[29392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506617, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:17.106940 31756 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:50:17.107372 31756 webserver.cc:460] Webserver started at http://127.7.132.131:34965/ using document root <none> and password file <none>
I20260812 03:50:17.107568 31756 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:17.107609 31756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:17.108783 31756 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:17.109439 31784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:17.109615 31756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:17.109680 31756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "bf13b7ffe2724dd78168c11ebafe0a0e"
format_stamp: "Formatted at 2026-08-12 03:50:15 on dist-test-slave-39l8"
server_key: "7a90a8d2c0f012a61627763d16714f95"
server_key_iv: "b81e0d6650a1552ebeec8e016170b27f"
server_key_version: "encryptionkey@0"
I20260812 03:50:17.110003 31756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:17.125910 31756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:17.127964 31756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:17.128069 31756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:17.128273 31756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:17.128544 31756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:17.128576 31756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:17.128594 31756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:17.128636 31756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:17.134445 31756 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:44405
I20260812 03:50:17.134495 31977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:44405 every 8 connection(s)
I20260812 03:50:17.134809 31756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:50:17.135776 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:17.135402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:33791)
0812 03:50:17.135556 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:17.135621 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
W20260812 03:50:17.136165 31978 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:33511 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:33511: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
I20260812 03:50:17.136452 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:17.136366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:58761)
0812 03:50:17.136411 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:17.136436 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:50:17.136709 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:17.136662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:34107)
0812 03:50:17.136681 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:17.136698 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260812 03:50:17.136840 31978 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:50:17.139308 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 31756
I20260812 03:50:17.938447 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:17.938222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:48949)
0812 03:50:17.938345 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:17.938376 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260812 03:50:18.007828 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.007642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:56581)
0812 03:50:18.007752 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:18.007785 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
I20260812 03:50:18.055974 31987 raft_consensus.cc:482] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:50:18.056097 31987 raft_consensus.cc:504] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:18.056501 31987 leader_election.cc:288] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 652251f89c2a4ed388def07ff2d2ad26 (127.7.132.130:44359), bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405)
Aug 12 03:50:18 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
Aug 12 03:50:18 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506616, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 03:50:18.066355 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.056621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:44359 (local address 127.7.132.129:56561)
0812 03:50:18.056674 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:18.056684 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:18.056914 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:18.057310 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:18.057312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:18.057693 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:18.057911 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:18.057917 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:18.059024 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:18.059024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:18.059804 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:18.059805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:18.060594 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:18.060595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:18.060661 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:18.061192 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:18.061197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:18.062571 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:18.065122 (+ 2551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:18.065123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:18.065124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:18.065289 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:18.065532 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:18.065533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:18.065534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:18.065598 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:18.065958 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:18.065959 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:18.066001 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:50:18.066090 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:50:18.066255 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":31}
I20260812 03:50:18.066380 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.056722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:56561 (local address 127.7.132.130:44359)
0812 03:50:18.056972 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:18.056980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:18.056997 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:50:18.057099 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:18.057100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:18.057139 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:18.057242 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:18.058020 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:18.058904 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:18.059906 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:18.060370 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:18.060488 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:18.062681 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:18.062684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:18.062684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:18.062693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:18.065003 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:18.065373 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:18.065376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:18.065376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:18.065463 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:18.065669 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:18.065670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:18.065670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:18.065881 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:18.065964 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:18.066153 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 03:50:18.066283 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":63,"threads_started":1}
I20260812 03:50:18.066401 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.056793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:44571 (local address 127.7.132.131:44405)
0812 03:50:18.056908 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:18.056920 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:18.056943 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:50:18.057489 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:18.057491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:18.057551 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:18.057679 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:18.058019 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:18.058906 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:18.059913 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:18.060324 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:18.060456 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:18.063091 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:18.063094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:18.063094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:18.063106 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:18.064720 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:18.065156 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:18.065159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:18.065160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:18.065249 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:18.065506 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:18.065507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:18.065508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:18.065746 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:18.065842 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:18.066134 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 03:50:18.066286 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":55,"threads_started":1}
I20260812 03:50:18.066469 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.056684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:44405 (local address 127.7.132.129:44571)
0812 03:50:18.056800 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:18.056812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:18.056874 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:18.057668 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:18.057669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:18.057822 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:18.057911 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:18.057916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:18.059009 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:18.059010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:18.059802 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:18.059804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:18.060435 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:18.060436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:18.060549 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:18.061144 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:18.061153 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:18.062964 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:18.064818 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:18.064825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:18.064827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:18.065048 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:18.065329 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:18.065330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:18.065330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:18.065438 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:18.065836 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:18.065843 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:18.065969 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 03:50:18.066065 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:50:18.066335 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":169,"thread_start_us":68,"threads_started":1}
I20260812 03:50:18.066885 31681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0474b5d4e174b41a8557d6a4883f00a" candidate_uuid: "aec81d6ccf204ed88c5bc182826cdec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "652251f89c2a4ed388def07ff2d2ad26" is_pre_election: true
I20260812 03:50:18.067030 31681 raft_consensus.cc:2454] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aec81d6ccf204ed88c5bc182826cdec1 in term 1.
W20260812 03:50:18.067162 31351 leader_election.cc:341] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405): Not found: Tablet not found: f0474b5d4e174b41a8557d6a4883f00a
I20260812 03:50:18.067313 31354 leader_election.cc:302] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652251f89c2a4ed388def07ff2d2ad26, aec81d6ccf204ed88c5bc182826cdec1; no voters: bf13b7ffe2724dd78168c11ebafe0a0e
I20260812 03:50:18.067431 31987 raft_consensus.cc:2781] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:50:18.067477 31987 raft_consensus.cc:482] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:50:18.067492 31987 raft_consensus.cc:3037] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:50:18.068562 31987 raft_consensus.cc:504] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:18.068701 31987 leader_election.cc:288] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [CANDIDATE]: Term 2 election: Requested vote from peers 652251f89c2a4ed388def07ff2d2ad26 (127.7.132.130:44359), bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405)
I20260812 03:50:18.068970 31681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0474b5d4e174b41a8557d6a4883f00a" candidate_uuid: "aec81d6ccf204ed88c5bc182826cdec1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "652251f89c2a4ed388def07ff2d2ad26"
W20260812 03:50:18.069020 31351 leader_election.cc:341] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405): Not found: Tablet not found: f0474b5d4e174b41a8557d6a4883f00a
I20260812 03:50:18.069089 31681 raft_consensus.cc:3037] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:50:18.070199 31681 raft_consensus.cc:2454] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aec81d6ccf204ed88c5bc182826cdec1 in term 2.
I20260812 03:50:18.070389 31354 leader_election.cc:302] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 652251f89c2a4ed388def07ff2d2ad26, aec81d6ccf204ed88c5bc182826cdec1; no voters: bf13b7ffe2724dd78168c11ebafe0a0e
I20260812 03:50:18.070478 31987 raft_consensus.cc:2781] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:50:18.070598 31987 raft_consensus.cc:686] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [term 2 LEADER]: Becoming Leader. State: Replica: aec81d6ccf204ed88c5bc182826cdec1, State: Running, Role: LEADER
I20260812 03:50:18.070741 31987 consensus_queue.cc:237] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:18.070990 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.070848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:42931)
0812 03:50:18.070937 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:18.070962 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 03:50:18.137329 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.137130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:36097)
0812 03:50:18.137262 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:18.137297 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260812 03:50:18.592170 31681 raft_consensus.cc:1261] T f0474b5d4e174b41a8557d6a4883f00a P 652251f89c2a4ed388def07ff2d2ad26 [term 2 FOLLOWER]: Refusing update from remote peer aec81d6ccf204ed88c5bc182826cdec1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 03:50:18.592542 32000 consensus_queue.cc:1035] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 03:50:18.593438 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.593291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:46997)
0812 03:50:18.593370 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:18.593404 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68,"thread_start_us":39,"threads_started":1}
I20260812 03:50:18.594439 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.594237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:41805)
0812 03:50:18.594365 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:18.594400 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260812 03:50:18.594442 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:18.594359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:35181)
0812 03:50:18.594396 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:18.594422 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 03:50:18.594523 31351 consensus_peers.cc:487] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1 -> Peer bf13b7ffe2724dd78168c11ebafe0a0e (127.7.132.131:44405): Couldn't send request to peer bf13b7ffe2724dd78168c11ebafe0a0e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0474b5d4e174b41a8557d6a4883f00a. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:50:19.068732 32015 ts_tablet_manager.cc:927] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Initiating tablet copy from peer aec81d6ccf204ed88c5bc182826cdec1 (127.7.132.129:45627)
I20260812 03:50:19.068974 32015 tablet_copy_client.cc:316] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: tablet copy: Beginning tablet copy session from remote peer at address 127.7.132.129:45627
Aug 12 03:50:19 dist-test-slave-39l8 krb5kdc[29392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506617, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:50:19.076767 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:19.069387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:52671 (local address 127.7.132.129:45627)
0812 03:50:19.069594 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:19.069599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:19.069616 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:50:19.069668 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:19.069670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:19.069687 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:19.069776 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:19.070221 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:19.070905 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:19.071635 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:19.071924 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:19.071974 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:19.074018 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:19.074026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:19.074026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:19.074033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:19.075756 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:19.076098 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:19.076101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:19.076101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:19.076157 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:19.076323 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:19.076323 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:19.076324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:19.076457 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:19.076517 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:19.076636 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 03:50:19.076702 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":128,"threads_started":1}
I20260812 03:50:19.076772 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:19.069273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45627 (local address 127.7.132.131:52671)
0812 03:50:19.069429 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:19.069444 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:19.069502 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:19.069788 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:19.069794 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:19.070076 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:19.070146 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:19.070148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:19.070971 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:19.070989 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:19.071567 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:19.071569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:19.072069 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:19.072069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:19.072124 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:19.072560 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:19.072564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:19.073937 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:19.075826 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:19.075832 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:19.075834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:19.076028 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:19.076213 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:19.076214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:19.076214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:19.076271 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:19.076510 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:19.076516 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:19.076555 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:50:19.076615 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:50:19.076695 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
I20260812 03:50:19.077284 31496 tablet_copy_service.cc:140] P aec81d6ccf204ed88c5bc182826cdec1: Received BeginTabletCopySession request for tablet f0474b5d4e174b41a8557d6a4883f00a from peer bf13b7ffe2724dd78168c11ebafe0a0e ({username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:52671)
I20260812 03:50:19.077347 31496 tablet_copy_service.cc:161] P aec81d6ccf204ed88c5bc182826cdec1: Beginning new tablet copy session on tablet f0474b5d4e174b41a8557d6a4883f00a from peer bf13b7ffe2724dd78168c11ebafe0a0e at {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:52671: session id = bf13b7ffe2724dd78168c11ebafe0a0e-f0474b5d4e174b41a8557d6a4883f00a
I20260812 03:50:19.077937 31496 tablet_copy_source_session.cc:205] T f0474b5d4e174b41a8557d6a4883f00a P aec81d6ccf204ed88c5bc182826cdec1: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:50:19.078928 32015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0474b5d4e174b41a8557d6a4883f00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:19.081635 32015 tablet_copy_client.cc:791] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: tablet copy: Starting download of 0 data blocks...
I20260812 03:50:19.081797 32015 tablet_copy_client.cc:655] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: tablet copy: Starting download of 1 WAL segments...
I20260812 03:50:19.082567 32015 tablet_copy_client.cc:531] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:50:19.083288 32015 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Bootstrap starting.
I20260812 03:50:19.084761 32015 log.cc:824] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:19.087347 32015 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:19.087574 32015 tablet_bootstrap.cc:492] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Bootstrap complete.
I20260812 03:50:19.087736 32015 ts_tablet_manager.cc:1397] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260812 03:50:19.088642 32015 raft_consensus.cc:348] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:19.088799 32015 raft_consensus.cc:729] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf13b7ffe2724dd78168c11ebafe0a0e, State: Initialized, Role: FOLLOWER
I20260812 03:50:19.088929 32015 consensus_queue.cc:260] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "652251f89c2a4ed388def07ff2d2ad26" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44359 } } peers { permanent_uuid: "bf13b7ffe2724dd78168c11ebafe0a0e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 44405 } } peers { permanent_uuid: "aec81d6ccf204ed88c5bc182826cdec1" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45627 } }
I20260812 03:50:19.089166 32015 ts_tablet_manager.cc:1428] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 03:50:19.089423 31496 tablet_copy_service.cc:342] P aec81d6ccf204ed88c5bc182826cdec1: Request end of tablet copy session bf13b7ffe2724dd78168c11ebafe0a0e-f0474b5d4e174b41a8557d6a4883f00a received from {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:52671
I20260812 03:50:19.089473 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:19.089360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:33497)
0812 03:50:19.089412 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:19.089445 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:50:19.089479 31496 tablet_copy_service.cc:433] P aec81d6ccf204ed88c5bc182826cdec1: ending tablet copy session bf13b7ffe2724dd78168c11ebafe0a0e-f0474b5d4e174b41a8557d6a4883f00a on tablet f0474b5d4e174b41a8557d6a4883f00a with peer bf13b7ffe2724dd78168c11ebafe0a0e
I20260812 03:50:19.089771 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:19.089679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:34469)
0812 03:50:19.089710 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:19.089736 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 03:50:19.523672 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:19.523460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:58537)
0812 03:50:19.523591 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:19.523628 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":91,"threads_started":1}
I20260812 03:50:19.523658 31901 raft_consensus.cc:1203] T f0474b5d4e174b41a8557d6a4883f00a P bf13b7ffe2724dd78168c11ebafe0a0e [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260812 03:50:19.524130 32018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:19.524011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:55763)
0812 03:50:19.524063 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:19.524092 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:50:19.595110 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:19.594927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:57521)
0812 03:50:19.595032 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:19.595065 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
I20260812 03:50:20.524299 32026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:20.524075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:50757)
0812 03:50:20.524215 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:20.524252 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":59,"threads_started":1}
I20260812 03:50:20.524767 32029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:20.524556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:38855)
0812 03:50:20.524694 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:20.524726 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260812 03:50:20.595701 32030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:20.595515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:54647)
0812 03:50:20.595625 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:20.595656 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260812 03:50:21.525014 32031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:21.524822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:47701)
0812 03:50:21.524938 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:21.524969 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":56,"threads_started":1}
I20260812 03:50:21.525401 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:21.525203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:36317)
0812 03:50:21.525332 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:21.525366 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260812 03:50:21.596400 32035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:21.596189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:40495)
0812 03:50:21.596322 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:21.596354 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260812 03:50:22.525739 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:22.525509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:37211)
0812 03:50:22.525627 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:22.525668 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260812 03:50:22.526083 32039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:22.525848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:41639)
0812 03:50:22.525992 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:22.526035 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260812 03:50:22.526208 32036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:22.526070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:43741)
0812 03:50:22.526117 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:22.526175 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 03:50:22.597246 32041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:22.596845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:54245)
0812 03:50:22.597140 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:22.597182 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":98,"threads_started":1}
I20260812 03:50:23.526752 32042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:23.526466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:34471)
0812 03:50:23.526614 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:23.526661 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":93,"threads_started":1}
I20260812 03:50:23.526844 32043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:23.526566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:47409)
0812 03:50:23.526746 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:23.526784 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260812 03:50:23.598060 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:23.597725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:55127)
0812 03:50:23.597972 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:23.598011 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":64,"threads_started":1}
I20260812 03:50:24.527455 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:24.527224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:59835)
0812 03:50:24.527364 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:24.527403 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260812 03:50:24.527455 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:24.527225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:50415)
0812 03:50:24.527365 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:24.527403 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260812 03:50:24.599023 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:24.598799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:33019)
0812 03:50:24.598937 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:24.598973 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260812 03:50:25.528159 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:25.527896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:58135)
0812 03:50:25.528063 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:25.528101 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260812 03:50:25.528194 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:25.527942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:57303)
0812 03:50:25.528112 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:25.528146 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260812 03:50:25.599666 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:25.599448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:43631)
0812 03:50:25.599583 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:25.599618 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":65,"threads_started":1}
I20260812 03:50:26.528779 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:26.528551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:33523)
0812 03:50:26.528687 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:26.528723 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":56,"threads_started":1}
I20260812 03:50:26.528872 32056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:26.528635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:38015)
0812 03:50:26.528775 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:26.528809 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260812 03:50:26.600446 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:26.600136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:46229)
0812 03:50:26.600354 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:26.600388 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":121,"threads_started":1}
I20260812 03:50:27.529687 32058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:27.529402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:50445)
0812 03:50:27.529568 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:27.529621 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
I20260812 03:50:27.529687 32060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:27.529402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:55449)
0812 03:50:27.529568 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:27.529621 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":77,"threads_started":1}
I20260812 03:50:27.601269 32061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:27.600986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:50833)
0812 03:50:27.601152 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:27.601199 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260812 03:50:28.530341 32063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:28.530121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:55827)
0812 03:50:28.530246 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:28.530280 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":56,"threads_started":1}
I20260812 03:50:28.530419 32065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:28.530194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:45333)
0812 03:50:28.530332 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:28.530380 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260812 03:50:28.602083 32066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:28.601796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:39895)
0812 03:50:28.601997 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:28.602035 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":58,"threads_started":1}
I20260812 03:50:29.531175 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:29.530958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:49435)
0812 03:50:29.531095 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:29.531131 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260812 03:50:29.531188 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:29.530990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:46629)
0812 03:50:29.531112 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:29.531147 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260812 03:50:29.602790 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:29.602523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:53543)
0812 03:50:29.602665 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:29.602735 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260812 03:50:30.531790 32073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:30.531574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:46511)
0812 03:50:30.531704 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:30.531743 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260812 03:50:30.531859 32074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:30.531675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:57345)
0812 03:50:30.531793 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:30.531821 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260812 03:50:30.603422 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:30.603189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:42457)
0812 03:50:30.603340 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:30.603374 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260812 03:50:31.532574 32077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:31.532337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.129:37827)
0812 03:50:31.532493 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:31.532525 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260812 03:50:31.532574 32076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:31.532349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.131:60387)
0812 03:50:31.532496 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:31.532531 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260812 03:50:31.604115 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:31.603864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:33511 (local address 127.7.132.130:58035)
0812 03:50:31.603999 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:31.604035 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:33511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260812 03:50:32.146569 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 31270
I20260812 03:50:32.158888 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 31546
I20260812 03:50:32.170943 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 31756
2026-08-12T03:50:32Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18878 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:50:32 dist-test-slave-39l8 krb5kdc[32085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:50:32 dist-test-slave-39l8 krb5kdc[32085](info): set up 2 sockets
Aug 12 03:50:32 dist-test-slave-39l8 krb5kdc[32085](info): commencing operation
krb5kdc: starting...
W20260812 03:50:33.382365 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:50:33 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:50:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:50:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:50:33.496852 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43005
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:33.578284 32102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:33.578462 32102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:33.578482 32102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:33.579604 32102 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:50:33.579643 32102 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:50:33.579655 32102 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:50:33.579666 32102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:33.579716 32102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:33.579734 32102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:33.579756 32102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:33.579783 32102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:33.581223 32102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:33.581472 32102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:33.581696 32102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:33.586686 32112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:33.586823 32102 server_base.cc:1034] running on GCE node
W20260812 03:50:33.586787 32113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:33.586804 32115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:33.587225 32102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:33.587504 32102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:33.588660 32102 hybrid_clock.cc:648] HybridClock initialized: now 1786506633588643 us; error 28 us; skew 500 ppm
Aug 12 03:50:33 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506633, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:33.590960 32102 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:33.591413 32102 webserver.cc:460] Webserver started at http://127.7.132.190:46489/ using document root <none> and password file <none>
I20260812 03:50:33.591630 32102 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:33.591713 32102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:33.591876 32102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:33.592903 32102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "6ace52b60fdc4485be60a510b4a93f40"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "3d7bbde7e389f2dec0717b0388958476"
server_key_iv: "f2fb9866e7afb257b27daa7923537b58"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.593271 32102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "6ace52b60fdc4485be60a510b4a93f40"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "3d7bbde7e389f2dec0717b0388958476"
server_key_iv: "f2fb9866e7afb257b27daa7923537b58"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.594561 32102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 03:50:33.595363 32122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:33.595541 32102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:50:33.595655 32102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "6ace52b60fdc4485be60a510b4a93f40"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "3d7bbde7e389f2dec0717b0388958476"
server_key_iv: "f2fb9866e7afb257b27daa7923537b58"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.595726 32102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:33.606954 32102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:33.608675 32102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:33.608829 32102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:33.613095 32102 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:43005
I20260812 03:50:33.613125 32211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:43005 every 8 connection(s)
I20260812 03:50:33.613488 32102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:33.614351 32212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:33.616609 32212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Bootstrap starting.
I20260812 03:50:33.617134 32212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:33.617384 32212 log.cc:824] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:33.617986 32212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: No bootstrap required, opened a new log
I20260812 03:50:33.619638 32212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:50:33.619774 32212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:33.619797 32212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ace52b60fdc4485be60a510b4a93f40, State: Initialized, Role: FOLLOWER
I20260812 03:50:33.619925 32212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:50:33.620002 32212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:50:33.620024 32212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:50:33.620050 32212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:33.620538 32212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:50:33.620620 32212 leader_election.cc:302] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ace52b60fdc4485be60a510b4a93f40; no voters:
I20260812 03:50:33.620779 32212 leader_election.cc:288] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:50:33.620806 32216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:33.620949 32216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 1 LEADER]: Becoming Leader. State: Replica: 6ace52b60fdc4485be60a510b4a93f40, State: Running, Role: LEADER
I20260812 03:50:33.621052 32212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:33.621106 32216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:50:33.621673 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 32102
I20260812 03:50:33.621812 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:50:33.622051 7698 external_mini_cluster.cc:1428] Setting key 175197cdc9a3d8f4ea5b5129a2bfae5c
I20260812 03:50:33.622331 32217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ace52b60fdc4485be60a510b4a93f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } } }
I20260812 03:50:33.622336 32219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ace52b60fdc4485be60a510b4a93f40. Latest consensus state: current_term: 1 leader_uuid: "6ace52b60fdc4485be60a510b4a93f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } } }
I20260812 03:50:33.622442 32217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:33.622460 32219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:33.622862 32247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:33.623634 32247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:50:33.625180 32247 catalog_manager.cc:1347] Generated new cluster ID: 4cf188fc27fa4f458fc6aff7e19a75c1
I20260812 03:50:33.625243 32247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:50:33 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:33.632906 32249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:33.622697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33588 (local address 127.7.132.190:43005)
0812 03:50:33.622981 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:33.622992 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:33.623035 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:50:33.623162 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:33.623169 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:33.623226 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:33.623386 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:33.624088 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.625425 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.626157 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.626568 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.626696 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:33.629018 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:33.629027 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:33.629028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:33.629039 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:33.631344 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:33.631713 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:33.631716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:33.631716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:33.631824 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:33.632047 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:33.632048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:33.632048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:33.632283 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:33.632433 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:33.632543 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 03:50:33.632697 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":64,"threads_started":1}
I20260812 03:50:33.641433 32247 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:50:33.641980 32247 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:50:33.653708 32247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Generated new TSK 0
I20260812 03:50:33.653904 32247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:50:33.687820 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43005
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:33.787719 32263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:33.787889 32263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:33.787910 32263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:33.788928 32263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:33.788966 32263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:33.789014 32263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:50:33.790355 32263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:33.790585 32263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:33.790833 32263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:33.795691 32271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:33.795688 32274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:33.795836 32263 server_base.cc:1034] running on GCE node
W20260812 03:50:33.795691 32272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:33.796068 32263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:33.796310 32263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:33.797469 32263 hybrid_clock.cc:648] HybridClock initialized: now 1786506633797441 us; error 28 us; skew 500 ppm
Aug 12 03:50:33 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506633, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:33.800197 32263 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:50:33.800622 32263 webserver.cc:460] Webserver started at http://127.7.132.129:40805/ using document root <none> and password file <none>
I20260812 03:50:33.800851 32263 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:33.800915 32263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:33.801035 32263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:33.801973 32263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "37ede9777b894344ba4330905bad13ce"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "8d2e1f18641cd1b08ed053f1b3178530"
server_key_iv: "8312ebe594b42b7c402d3ec36da4a14e"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.802317 32263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "37ede9777b894344ba4330905bad13ce"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "8d2e1f18641cd1b08ed053f1b3178530"
server_key_iv: "8312ebe594b42b7c402d3ec36da4a14e"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.803660 32263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:50:33.804425 32286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:33.804597 32263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:33.804702 32263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "37ede9777b894344ba4330905bad13ce"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "8d2e1f18641cd1b08ed053f1b3178530"
server_key_iv: "8312ebe594b42b7c402d3ec36da4a14e"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.804793 32263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:33.820504 32263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:33.823100 32263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:33.823261 32263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:33.823495 32263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:33.823792 32263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:33.823848 32263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:33.823886 32263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:33.823912 32263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:33.829591 32263 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45463
I20260812 03:50:33.829643 32467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45463 every 8 connection(s)
I20260812 03:50:33.829908 32263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:50:33.833493 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 32263
I20260812 03:50:33.833613 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:50:33.833832 7698 external_mini_cluster.cc:1428] Setting key a70435324e36fb9aa4fa79db993daf1a
Aug 12 03:50:33 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506633, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:33.839636 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:33.830546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:49719)
0812 03:50:33.830725 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:33.830763 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:33.831223 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:33.831450 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:33.831453 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:33.831977 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:33.832200 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:33.832204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.832843 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.832844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:33.833688 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:33.833692 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.834179 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.834181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:33.834327 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:33.834835 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:33.834843 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:33.836483 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:33.838199 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:33.838205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:33.838207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:33.838417 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:33.838631 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:33.838632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:33.838632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:33.838794 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:33.839103 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:33.839108 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:33.839263 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 03:50:33.839372 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 03:50:33.839486 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260812 03:50:33.839651 32249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:33.830583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:49719 (local address 127.7.132.190:43005)
0812 03:50:33.830625 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:33.830627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:33.830844 (+ 217us) server_negotiation.cc:407] Connection header received
0812 03:50:33.831303 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:33.831304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:33.831317 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:33.831372 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:33.832278 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.832763 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.833769 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.834061 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.834147 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:33.836586 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:33.836587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:33.836588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:33.836598 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:33.838091 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:33.838494 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:33.838496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:33.838497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:33.838560 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:33.838864 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:33.838865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:33.838866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:33.839025 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:33.839114 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:33.839450 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 03:50:33.839545 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:50:33.840216 32469 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43005
I20260812 03:50:33.840329 32469 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:33.840517 32469 heartbeater.cc:511] Master 127.7.132.190:43005 requested a full tablet report, sending...
I20260812 03:50:33.841178 32156 ts_manager.cc:194] Registered new tserver with Master: 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463)
I20260812 03:50:33.842144 32156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:49719
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:50:33.858688 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43005
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:33.938277 32476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:33.938462 32476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:33.938484 32476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:33.939529 32476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:33.939563 32476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:33.939595 32476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:50:33.940907 32476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:33.941082 32476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:33.941248 32476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:33.946076 32483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:33.946154 32476 server_base.cc:1034] running on GCE node
W20260812 03:50:33.946076 32485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:33.946110 32481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:33.946529 32476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:33.946759 32476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:33.947923 32476 hybrid_clock.cc:648] HybridClock initialized: now 1786506633947913 us; error 24 us; skew 500 ppm
Aug 12 03:50:33 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506633, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:33.950309 32476 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:50:33.950740 32476 webserver.cc:460] Webserver started at http://127.7.132.130:45215/ using document root <none> and password file <none>
I20260812 03:50:33.950999 32476 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:33.951066 32476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:33.951212 32476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:33.952200 32476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "58e444ce901f4511b9558675d03aed8a"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "ad93e1e9502c435ff111f88408ea0674"
server_key_iv: "21a60575ee09e84f62b228346bae6896"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.952548 32476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "58e444ce901f4511b9558675d03aed8a"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "ad93e1e9502c435ff111f88408ea0674"
server_key_iv: "21a60575ee09e84f62b228346bae6896"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.953867 32476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:50:33.954582 32496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:33.954787 32476 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:33.954893 32476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "58e444ce901f4511b9558675d03aed8a"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "ad93e1e9502c435ff111f88408ea0674"
server_key_iv: "21a60575ee09e84f62b228346bae6896"
server_key_version: "encryptionkey@0"
I20260812 03:50:33.954975 32476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:33.962834 32476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:33.964488 32476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:33.964654 32476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:33.964864 32476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:33.965155 32476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:33.965211 32476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:33.965252 32476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:33.965279 32476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:33.971105 32476 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:34857
I20260812 03:50:33.971154 32682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:34857 every 8 connection(s)
I20260812 03:50:33.971479 32476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:50:33.973053 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 32476
I20260812 03:50:33.973165 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:50:33.973379 7698 external_mini_cluster.cc:1428] Setting key 87b9cbc37a066975db3bd2ae22c02c5e
Aug 12 03:50:33 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506633, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:33.981994 32249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:33.972257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:46029 (local address 127.7.132.190:43005)
0812 03:50:33.972325 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:33.972327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:33.972493 (+ 166us) server_negotiation.cc:407] Connection header received
0812 03:50:33.972962 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:33.972964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:33.972977 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:33.973064 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:33.974256 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.975051 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.976111 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.976291 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.976347 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:33.978930 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:33.978932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:33.978932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:33.978939 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:33.980449 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:33.981015 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:33.981016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:33.981017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:33.981080 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:33.981383 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:33.981383 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:33.981383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:33.981482 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:33.981530 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:33.981869 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 03:50:33.981934 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:50:33.982261 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:33.972207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:46029)
0812 03:50:33.972366 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:33.972405 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:33.972870 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:33.973114 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:33.973117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:33.973896 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:33.974163 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:33.974170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.975138 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.975139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:33.976014 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:33.976018 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:33.976376 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:33.976378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:33.976521 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:33.977032 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:33.977042 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:33.978839 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:33.980571 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:33.980580 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:33.980582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:33.980931 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:33.981154 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:33.981155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:33.981155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:33.981303 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:33.981556 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:33.981563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:33.981737 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 03:50:33.981854 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:50:33.982027 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260812 03:50:33.982849 32683 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43005
I20260812 03:50:33.982945 32683 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:33.983144 32683 heartbeater.cc:511] Master 127.7.132.190:43005 requested a full tablet report, sending...
I20260812 03:50:33.983597 32156 ts_manager.cc:194] Registered new tserver with Master: 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857)
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
I20260812 03:50:33.983976 32156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:46029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:50:33.999418 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43005
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:34.082748 32692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:34.082984 32692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:34.083019 32692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:34.084101 32692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:34.084177 32692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:34.084224 32692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:50:34.085690 32692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:34.085963 32692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:34.086189 32692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:34.091557 32702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:34.091586 32700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:34.091558 32699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:34.091810 32692 server_base.cc:1034] running on GCE node
I20260812 03:50:34.092007 32692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:34.092221 32692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:34.093365 32692 hybrid_clock.cc:648] HybridClock initialized: now 1786506634093356 us; error 27 us; skew 500 ppm
Aug 12 03:50:34 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506634, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:34.095917 32692 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:50:34.096321 32692 webserver.cc:460] Webserver started at http://127.7.132.131:42775/ using document root <none> and password file <none>
I20260812 03:50:34.096530 32692 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:34.096573 32692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:34.096712 32692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:50:34.097681 32692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "8daa756e125c4a758a2d0dc573c8317e"
format_stamp: "Formatted at 2026-08-12 03:50:34 on dist-test-slave-39l8"
server_key: "52f6a40660ec734179cc5dddb247ad05"
server_key_iv: "955ae6b51a6f1b84d576d69f062f7a6b"
server_key_version: "encryptionkey@0"
I20260812 03:50:34.098044 32692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "8daa756e125c4a758a2d0dc573c8317e"
format_stamp: "Formatted at 2026-08-12 03:50:34 on dist-test-slave-39l8"
server_key: "52f6a40660ec734179cc5dddb247ad05"
server_key_iv: "955ae6b51a6f1b84d576d69f062f7a6b"
server_key_version: "encryptionkey@0"
I20260812 03:50:34.099503 32692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:50:34.100188 32711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:34.100368 32692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:34.100437 32692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "8daa756e125c4a758a2d0dc573c8317e"
format_stamp: "Formatted at 2026-08-12 03:50:34 on dist-test-slave-39l8"
server_key: "52f6a40660ec734179cc5dddb247ad05"
server_key_iv: "955ae6b51a6f1b84d576d69f062f7a6b"
server_key_version: "encryptionkey@0"
I20260812 03:50:34.100538 32692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:34.117101 32692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:34.118901 32692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:34.119045 32692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:34.119259 32692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:34.119534 32692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:34.119568 32692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:34.119606 32692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:34.119630 32692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:34.125872 32692 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:40681
I20260812 03:50:34.125967 422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:40681 every 8 connection(s)
I20260812 03:50:34.126240 32692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:50:34 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506634, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:34.134191 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 32692
I20260812 03:50:34.134289 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:50:34.134501 7698 external_mini_cluster.cc:1428] Setting key 78dc8e2c4ac6596b53e677f7986d872f
I20260812 03:50:34.135176 32249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.126964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:45233 (local address 127.7.132.190:43005)
0812 03:50:34.127010 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:34.127012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:34.127202 (+ 190us) server_negotiation.cc:407] Connection header received
0812 03:50:34.127711 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:34.127712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:34.127725 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:34.127774 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:34.128721 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.129182 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.130097 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.130298 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.130367 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.132394 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:34.132395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:34.132395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:34.132402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:34.133642 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:34.133970 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:34.133970 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:34.133971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:34.134032 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:34.134400 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:34.134401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:34.134401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:34.134515 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:34.134605 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:34.134975 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 03:50:34.135066 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:50:34.135361 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.126924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:45233)
0812 03:50:34.127073 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:34.127115 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:34.127605 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:34.127852 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:34.127855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:34.128427 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:34.128601 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.128606 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.129274 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.129275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.130005 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.130007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.130405 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.130406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.130527 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.130983 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:34.130992 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:34.132303 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:34.133710 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:34.133717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:34.133718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:34.133908 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:34.134110 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:34.134111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:34.134111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:34.134276 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:34.134611 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:34.134619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:34.134813 (+ 194us) client_negotiation.cc:769] Sending connection context
0812 03:50:34.134948 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 03:50:34.135114 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
I20260812 03:50:34.135860 425 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43005
I20260812 03:50:34.135934 425 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:34.136060 425 heartbeater.cc:511] Master 127.7.132.190:43005 requested a full tablet report, sending...
I20260812 03:50:34.136373 32156 ts_manager.cc:194] Registered new tserver with Master: 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
I20260812 03:50:34.136718 32156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:45233
I20260812 03:50:34.146374 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:50:34.152889 32249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.147712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.7.132.190:43005)
0812 03:50:34.147752 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:34.147754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:34.147887 (+ 133us) server_negotiation.cc:407] Connection header received
0812 03:50:34.147927 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:34.147928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:34.147936 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:34.147970 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:34.148455 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.148901 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.149540 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.149701 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.149755 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.150886 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:34.150887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:34.150887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:34.150892 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:34.151969 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:34.152280 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:34.152280 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:34.152281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:34.152326 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:34.152490 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:34.152491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:34.152491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:34.152612 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:34.152682 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:34.152778 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:50:34.152832 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:50:34.154469 32156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:50:34.156122 32156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:50:34.162577 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.158593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:40681 (local address 127.0.0.1:53408)
0812 03:50:34.158963 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:34.158972 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:34.159022 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:34.159275 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:34.159276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:34.159279 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:34.159354 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.159357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.160222 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.160223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.161377 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.161380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.162234 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.162235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.162323 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.162336 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 03:50:34.162448 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:50:34.162512 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260812 03:50:34.162590 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.158673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.7.132.131:40681)
0812 03:50:34.158880 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:34.158885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:34.159035 (+ 150us) server_negotiation.cc:407] Connection header received
0812 03:50:34.159108 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:34.159110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:34.159125 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:34.159205 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:34.159437 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.160101 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.161476 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.162145 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.162227 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.162284 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:34.162476 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 03:50:34.162523 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":133,"threads_started":1}
I20260812 03:50:34.162964 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.158384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45463 (local address 127.0.0.1:37722)
0812 03:50:34.158680 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:34.158748 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:34.158838 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:34.159339 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:34.159340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:34.159342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:34.159380 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.159381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.160703 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.160704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.161673 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.161675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.162560 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.162561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.162620 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.162624 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:50:34.162850 (+ 226us) client_negotiation.cc:240] Negotiation successful
0812 03:50:34.162895 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":57,"threads_started":1}
I20260812 03:50:34.163098 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.158501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.7.132.130:34857)
0812 03:50:34.158932 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:34.158938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:34.158949 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:50:34.158992 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:34.158995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:34.159011 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:34.159174 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:34.159424 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.160581 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.161995 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.162732 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.162796 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.162848 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:34.162943 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 03:50:34.162983 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":74,"threads_started":1}
I20260812 03:50:34.163079 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.158384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34857 (local address 127.0.0.1:34630)
0812 03:50:34.158754 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:34.158763 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:34.158842 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:34.159265 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:34.159268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:34.159271 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:34.159338 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.159340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.160726 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.160727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.161906 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.161909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.162800 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.162801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.162863 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.162867 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:50:34.162928 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:50:34.162967 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":4,"thread_start_us":70,"threads_started":1}
I20260812 03:50:34.163436 320 tablet_service.cc:1467] Processing CreateTablet for tablet e25ea77152d94b7eaf03539a45e10485 (DEFAULT_TABLE table=test-table [id=1810df33822f435f84d2e86459492fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:34.163743 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.158486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.7.132.129:45463)
0812 03:50:34.159161 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:34.159165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:34.159175 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:50:34.159213 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:34.159215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:34.159239 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:34.159456 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:34.159531 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.160599 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.161756 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.162448 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.163428 (+ 980us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.163586 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:34.163611 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 03:50:34.163660 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":95,"threads_started":1}
I20260812 03:50:34.163820 320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e25ea77152d94b7eaf03539a45e10485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:34.164196 32571 tablet_service.cc:1467] Processing CreateTablet for tablet e25ea77152d94b7eaf03539a45e10485 (DEFAULT_TABLE table=test-table [id=1810df33822f435f84d2e86459492fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:34.164469 32571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e25ea77152d94b7eaf03539a45e10485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:34.164418 32356 tablet_service.cc:1467] Processing CreateTablet for tablet e25ea77152d94b7eaf03539a45e10485 (DEFAULT_TABLE table=test-table [id=1810df33822f435f84d2e86459492fa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:34.164754 32356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e25ea77152d94b7eaf03539a45e10485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:34.167027 453 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Bootstrap starting.
I20260812 03:50:34.167553 454 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: Bootstrap starting.
I20260812 03:50:34.167582 453 tablet_bootstrap.cc:654] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:34.167945 456 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Bootstrap starting.
I20260812 03:50:34.168208 454 tablet_bootstrap.cc:654] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:34.168185 453 log.cc:824] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:34.168553 456 tablet_bootstrap.cc:654] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:34.168779 454 log.cc:824] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:34.168973 453 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: No bootstrap required, opened a new log
I20260812 03:50:34.169044 453 ts_tablet_manager.cc:1397] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:50:34.169116 456 log.cc:824] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:34.169567 454 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: No bootstrap required, opened a new log
I20260812 03:50:34.169685 454 ts_tablet_manager.cc:1397] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:50:34.169860 456 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: No bootstrap required, opened a new log
I20260812 03:50:34.169930 456 ts_tablet_manager.cc:1397] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:50:34.171065 453 raft_consensus.cc:348] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.171250 453 raft_consensus.cc:374] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:34.171290 453 raft_consensus.cc:729] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e444ce901f4511b9558675d03aed8a, State: Initialized, Role: FOLLOWER
I20260812 03:50:34.171437 453 consensus_queue.cc:260] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.171412 456 raft_consensus.cc:348] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.171640 456 raft_consensus.cc:374] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:34.171694 456 raft_consensus.cc:729] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ede9777b894344ba4330905bad13ce, State: Initialized, Role: FOLLOWER
I20260812 03:50:34.171723 453 ts_tablet_manager.cc:1428] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:50:34.171658 454 raft_consensus.cc:348] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.171818 454 raft_consensus.cc:374] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:34.171857 32683 heartbeater.cc:503] Master 127.7.132.190:43005 was elected leader, sending a full tablet report...
I20260812 03:50:34.171860 454 raft_consensus.cc:729] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8daa756e125c4a758a2d0dc573c8317e, State: Initialized, Role: FOLLOWER
I20260812 03:50:34.171831 456 consensus_queue.cc:260] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.171981 454 consensus_queue.cc:260] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.172125 456 ts_tablet_manager.cc:1428] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:50:34.172130 32469 heartbeater.cc:503] Master 127.7.132.190:43005 was elected leader, sending a full tablet report...
I20260812 03:50:34.172336 454 ts_tablet_manager.cc:1428] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:50:34.172458 425 heartbeater.cc:503] Master 127.7.132.190:43005 was elected leader, sending a full tablet report...
W20260812 03:50:34.222064 32688 tablet.cc:2369] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:50:34.268684 462 raft_consensus.cc:482] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:50:34.268828 462 raft_consensus.cc:504] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.269165 462 leader_election.cc:288] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463), 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
I20260812 03:50:34.272933 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.269586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:48257 (local address 127.7.132.131:40681)
0812 03:50:34.269646 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:34.269648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:34.269809 (+ 161us) server_negotiation.cc:407] Connection header received
0812 03:50:34.269876 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:34.269877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:34.269887 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:34.269936 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:34.270065 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.270684 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.271943 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.272550 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.272609 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.272618 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:34.272814 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 03:50:34.272866 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:50:34.272955 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.269358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45463 (local address 127.7.132.130:34835)
0812 03:50:34.269425 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:34.269443 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:34.269514 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:34.269719 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:34.269721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:34.269723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:34.269798 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.269800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.270676 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.270676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.271859 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.271861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.272665 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.272666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.272750 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.272753 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:50:34.272804 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:50:34.272851 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":78}
I20260812 03:50:34.272995 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.269473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:34835 (local address 127.7.132.129:45463)
0812 03:50:34.269540 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:34.269542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:34.269554 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:50:34.269588 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:34.269588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:34.269599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:34.269665 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:34.269880 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.270600 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.271950 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.272528 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.272610 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.272618 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:34.272881 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 03:50:34.272926 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:50:34.272961 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.269508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:40681 (local address 127.7.132.130:48257)
0812 03:50:34.269717 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:34.269727 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:34.269794 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:34.269950 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:34.269951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:34.269952 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:34.269998 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.269999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.270777 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.270778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.271858 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:34.271861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.272666 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.272667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:34.272745 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.272749 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:50:34.272804 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 03:50:34.272851 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":9,"thread_start_us":121,"threads_started":1}
I20260812 03:50:34.273473 32394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e25ea77152d94b7eaf03539a45e10485" candidate_uuid: "58e444ce901f4511b9558675d03aed8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ede9777b894344ba4330905bad13ce" is_pre_election: true
I20260812 03:50:34.273483 351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e25ea77152d94b7eaf03539a45e10485" candidate_uuid: "58e444ce901f4511b9558675d03aed8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8daa756e125c4a758a2d0dc573c8317e" is_pre_election: true
I20260812 03:50:34.273594 351 raft_consensus.cc:2454] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58e444ce901f4511b9558675d03aed8a in term 0.
I20260812 03:50:34.273633 32394 raft_consensus.cc:2454] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58e444ce901f4511b9558675d03aed8a in term 0.
I20260812 03:50:34.273842 32498 leader_election.cc:302] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58e444ce901f4511b9558675d03aed8a, 8daa756e125c4a758a2d0dc573c8317e; no voters:
I20260812 03:50:34.274003 462 raft_consensus.cc:2781] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:50:34.274077 462 raft_consensus.cc:482] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:50:34.274096 462 raft_consensus.cc:3037] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:34.274874 462 raft_consensus.cc:504] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.275008 462 leader_election.cc:288] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [CANDIDATE]: Term 1 election: Requested vote from peers 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463), 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
I20260812 03:50:34.275200 32394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e25ea77152d94b7eaf03539a45e10485" candidate_uuid: "58e444ce901f4511b9558675d03aed8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ede9777b894344ba4330905bad13ce"
I20260812 03:50:34.275260 351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e25ea77152d94b7eaf03539a45e10485" candidate_uuid: "58e444ce901f4511b9558675d03aed8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8daa756e125c4a758a2d0dc573c8317e"
I20260812 03:50:34.275295 32394 raft_consensus.cc:3037] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:34.275372 351 raft_consensus.cc:3037] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:34.276091 32394 raft_consensus.cc:2454] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58e444ce901f4511b9558675d03aed8a in term 1.
I20260812 03:50:34.276118 351 raft_consensus.cc:2454] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58e444ce901f4511b9558675d03aed8a in term 1.
I20260812 03:50:34.276269 32498 leader_election.cc:302] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58e444ce901f4511b9558675d03aed8a, 8daa756e125c4a758a2d0dc573c8317e; no voters:
I20260812 03:50:34.276381 462 raft_consensus.cc:2781] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:34.276518 462 raft_consensus.cc:686] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 1 LEADER]: Becoming Leader. State: Replica: 58e444ce901f4511b9558675d03aed8a, State: Running, Role: LEADER
I20260812 03:50:34.276638 462 consensus_queue.cc:237] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:34.277420 32156 catalog_manager.cc:5495] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 58e444ce901f4511b9558675d03aed8a (127.7.132.130). New cstate: current_term: 1 leader_uuid: "58e444ce901f4511b9558675d03aed8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } health_report { overall_health: HEALTHY } } }
I20260812 03:50:34.306212 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.303841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.7.132.130:34857)
0812 03:50:34.303884 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:34.303886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:34.303893 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:50:34.303945 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:34.303946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:34.303960 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:34.304015 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:50:34.304158 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.304613 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.305404 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:34.305605 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:34.305677 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:34.305849 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:50:34.305945 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:50:34.306040 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:34.306065 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 03:50:34.306084 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:50:34.307992 351 raft_consensus.cc:1261] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 1 FOLLOWER]: Refusing update from remote peer 58e444ce901f4511b9558675d03aed8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:34.307991 32394 raft_consensus.cc:1261] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Refusing update from remote peer 58e444ce901f4511b9558675d03aed8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:34.308362 462 consensus_queue.cc:1035] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:34.308557 469 consensus_queue.cc:1035] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:34.310818 474 mvcc.cc:205] Tried to move back new op lower bound from 7317531174122819584 to 7317531173997694976. Current Snapshot: MvccSnapshot[applied={T|T < 7317531174122819584}]
I20260812 03:50:34.311347 475 mvcc.cc:205] Tried to move back new op lower bound from 7317531174122819584 to 7317531173997694976. Current Snapshot: MvccSnapshot[applied={T|T < 7317531174122819584}]
I20260812 03:50:34.313026 32154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:34.313107 32154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:34.313876 32154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Sending DeleteTablet for 3 replicas of tablet e25ea77152d94b7eaf03539a45e10485
I20260812 03:50:34.314229 32356 tablet_service.cc:1514] Processing DeleteTablet for tablet e25ea77152d94b7eaf03539a45e10485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:34 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:37722
I20260812 03:50:34.314275 32571 tablet_service.cc:1514] Processing DeleteTablet for tablet e25ea77152d94b7eaf03539a45e10485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:34 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:34630
I20260812 03:50:34.314297 320 tablet_service.cc:1514] Processing DeleteTablet for tablet e25ea77152d94b7eaf03539a45e10485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:34 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:53408
I20260812 03:50:34.314574 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 32263
I20260812 03:50:34.314575 499 tablet_replica.cc:330] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: stopping tablet replica
I20260812 03:50:34.314586 496 tablet_replica.cc:330] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: stopping tablet replica
I20260812 03:50:34.320065 499 raft_consensus.cc:2229] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:50:34.320220 499 raft_consensus.cc:2258] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:34.326812 498 tablet_replica.cc:330] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: stopping tablet replica
I20260812 03:50:34.326960 498 raft_consensus.cc:2229] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:50:34.327107 498 raft_consensus.cc:2258] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:34.328665 498 ts_tablet_manager.cc:1905] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:34.331127 498 ts_tablet_manager.cc:1918] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:34.331243 498 log.cc:1196] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/e25ea77152d94b7eaf03539a45e10485
I20260812 03:50:34.331532 498 ts_tablet_manager.cc:1939] T e25ea77152d94b7eaf03539a45e10485 P 58e444ce901f4511b9558675d03aed8a: Deleting consensus metadata
I20260812 03:50:34.331955 32132 catalog_manager.cc:4841] TS 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857): tablet e25ea77152d94b7eaf03539a45e10485 (table test-table [id=1810df33822f435f84d2e86459492fa8]) successfully deleted
I20260812 03:50:34.334612 499 ts_tablet_manager.cc:1905] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:34.335968 499 ts_tablet_manager.cc:1918] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:34.336053 499 log.cc:1196] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/e25ea77152d94b7eaf03539a45e10485
I20260812 03:50:34.336428 499 ts_tablet_manager.cc:1939] T e25ea77152d94b7eaf03539a45e10485 P 8daa756e125c4a758a2d0dc573c8317e: Deleting consensus metadata
I20260812 03:50:34.336848 32124 catalog_manager.cc:4841] TS 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681): tablet e25ea77152d94b7eaf03539a45e10485 (table test-table [id=1810df33822f435f84d2e86459492fa8]) successfully deleted
W20260812 03:50:34.357074 32124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:45463 (error 108)
I20260812 03:50:34.357333 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 32476
I20260812 03:50:34.359898 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:34.357492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45463 (local address 127.0.0.1:37730)
0812 03:50:34.359794 (+ 2302us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:34.359839 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2265}
W20260812 03:50:34.360920 32124 catalog_manager.cc:4584] TS 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e25ea77152d94b7eaf03539a45e10485: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45463: connect: Connection refused (error 111)
I20260812 03:50:34.368803 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 32692
I20260812 03:50:34.390147 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 32102
I20260812 03:50:54.399282 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43005
--webserver_interface=127.7.132.190
--webserver_port=46489
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:50:54.478614 503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:54.478818 503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:54.478839 503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:54.479954 503 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:50:54.479992 503 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:50:54.480005 503 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:50:54.480017 503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:50:54.480070 503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:54.480088 503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:54.480111 503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:50:54.480140 503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:50:54.481601 503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=46489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:54.481853 503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:54.482090 503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:54.491072 516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:54.491072 512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:54.491096 513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:54.491173 503 server_base.cc:1034] running on GCE node
I20260812 03:50:54.491601 503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:54.491936 503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:54.493103 503 hybrid_clock.cc:648] HybridClock initialized: now 1786506654493089 us; error 32 us; skew 500 ppm
Aug 12 03:50:54 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506654, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:54.495620 503 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:50:54.496098 503 webserver.cc:460] Webserver started at http://127.7.132.190:46489/ using document root <none> and password file <none>
I20260812 03:50:54.496354 503 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:54.496446 503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:54.497785 503 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:54.498432 525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.498601 503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:50:54.498713 503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "6ace52b60fdc4485be60a510b4a93f40"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "3d7bbde7e389f2dec0717b0388958476"
server_key_iv: "f2fb9866e7afb257b27daa7923537b58"
server_key_version: "encryptionkey@0"
I20260812 03:50:54.499158 503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:54.523725 503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:54.525800 503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:54.525979 503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:54.530427 503 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:43005
I20260812 03:50:54.530479 610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:43005 every 8 connection(s)
I20260812 03:50:54.530980 503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:50:54.532464 612 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:50:54.533142 612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Bootstrap starting.
I20260812 03:50:54.535328 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 503
I20260812 03:50:54.535643 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:45463
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:43005
--webserver_port=40805
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:54.542438 612 log.cc:824] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:54.544399 612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:54.544749 612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Bootstrap complete.
I20260812 03:50:54.547188 612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:50:54.547456 612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ace52b60fdc4485be60a510b4a93f40, State: Initialized, Role: FOLLOWER
I20260812 03:50:54.547619 612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:50:54.547688 612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:50:54.547750 612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:50:54.547796 612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:50:54.548718 612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:50:54.548838 612 leader_election.cc:302] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ace52b60fdc4485be60a510b4a93f40; no voters:
I20260812 03:50:54.549006 612 leader_election.cc:288] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:50:54.549036 619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:50:54.549235 619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 2 LEADER]: Becoming Leader. State: Replica: 6ace52b60fdc4485be60a510b4a93f40, State: Running, Role: LEADER
I20260812 03:50:54.549291 612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:50:54.549393 619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:50:54.549713 622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6ace52b60fdc4485be60a510b4a93f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } } }
I20260812 03:50:54.549794 622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:54.550169 623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ace52b60fdc4485be60a510b4a93f40. Latest consensus state: current_term: 2 leader_uuid: "6ace52b60fdc4485be60a510b4a93f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } } }
I20260812 03:50:54.550243 623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:50:54.550411 636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:50:54.551223 636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:50:54.551376 636 catalog_manager.cc:1259] Loaded cluster ID: 4cf188fc27fa4f458fc6aff7e19a75c1
I20260812 03:50:54.551610 636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:50:54.552960 636 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:50:54.553202 636 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Loaded TSK: 0
I20260812 03:50:54.565680 636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Generated new TSK 1
I20260812 03:50:54.565758 636 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:50:54.621683 614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:54.621940 614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:54.621978 614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:54.623067 614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:54.623131 614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:54.623180 614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:50:54.624559 614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:45463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=40805
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:54.624821 614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:54.625059 614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:54.630765 664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:54.630777 659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:54.630761 660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:54.630975 614 server_base.cc:1034] running on GCE node
I20260812 03:50:54.631172 614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:54.631417 614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:54.632582 614 hybrid_clock.cc:648] HybridClock initialized: now 1786506654632558 us; error 41 us; skew 500 ppm
Aug 12 03:50:54 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506654, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:54.635068 614 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:50:54.635530 614 webserver.cc:460] Webserver started at http://127.7.132.129:40805/ using document root <none> and password file <none>
I20260812 03:50:54.635761 614 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:54.635816 614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:54.637240 614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:54.637975 675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.638144 614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:50:54.638248 614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "37ede9777b894344ba4330905bad13ce"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "8d2e1f18641cd1b08ed053f1b3178530"
server_key_iv: "8312ebe594b42b7c402d3ec36da4a14e"
server_key_version: "encryptionkey@0"
I20260812 03:50:54.638604 614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:54.655067 614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:54.657195 614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:54.657353 614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:54.657590 614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:54.658119 685 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:50:54.659622 614 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:50:54.659674 614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 03:50:54.659708 614 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:50:54.660447 614 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:50:54.660492 614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:50:54.660535 685 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Bootstrap starting.
I20260812 03:50:54.668111 614 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45463
I20260812 03:50:54.668618 614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:50:54.668987 859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45463 every 8 connection(s)
I20260812 03:50:54.671918 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 614
I20260812 03:50:54.672247 685 log.cc:824] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:54.672283 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:34857
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:43005
--webserver_port=45215
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:50:54.674674 685 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:50:54.674989 685 tablet_bootstrap.cc:492] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Bootstrap complete.
I20260812 03:50:54.676249 685 ts_tablet_manager.cc:1397] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
Aug 12 03:50:54 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506654, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:54.677977 685 raft_consensus.cc:348] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:54.678375 685 raft_consensus.cc:729] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ede9777b894344ba4330905bad13ce, State: Initialized, Role: FOLLOWER
I20260812 03:50:54.678568 685 consensus_queue.cc:260] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } }
I20260812 03:50:54.678958 685 ts_tablet_manager.cc:1428] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:50:54.681924 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.669638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:41059 (local address 127.7.132.190:43005)
0812 03:50:54.669853 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:54.669865 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:54.671002 (+ 1137us) server_negotiation.cc:407] Connection header received
0812 03:50:54.671671 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:54.671679 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:54.671738 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:54.671880 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:54.672813 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.674108 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.675678 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.676307 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.676874 (+ 567us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.678746 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:54.678750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:54.678750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:54.678761 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:54.680259 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.680688 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.680691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.680692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.680785 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.681041 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.681042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.681042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.681250 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:54.681339 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:54.681643 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 03:50:54.681756 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260812 03:50:54.681924 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.669309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:41059)
0812 03:50:54.670837 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:54.670880 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:54.671441 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:54.672017 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:54.672021 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:54.672418 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:54.672675 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.672682 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.674274 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.674275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.675585 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.675589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.676441 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.676443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.676596 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.677197 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:54.677205 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:54.678632 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:54.680358 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:54.680365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:54.680367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:54.680601 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:54.680858 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:54.680859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:54.680860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:54.680967 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:54.681330 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:54.681337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:54.681468 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 03:50:54.681583 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:50:54.681750 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1486,"thread_start_us":80,"threads_started":1}
I20260812 03:50:54.682504 860 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43005
I20260812 03:50:54.682585 860 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:54.682777 860 heartbeater.cc:511] Master 127.7.132.190:43005 requested a full tablet report, sending...
I20260812 03:50:54.683434 551 ts_manager.cc:194] Registered new tserver with Master: 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463)
I20260812 03:50:54.684582 551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:41059
I20260812 03:50:54.685521 860 heartbeater.cc:503] Master 127.7.132.190:43005 was elected leader, sending a full tablet report...
Aug 12 03:50:54 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506654, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.129@KRBTEST.COM
I20260812 03:50:54.692343 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.684020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45463 (local address 127.0.0.1:46150)
0812 03:50:54.684317 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:54.684359 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:54.684458 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:54.684659 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:54.684664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:54.685027 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:54.685095 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.685098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.686181 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.686182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.687104 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.687107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.687609 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.687610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.687694 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.688080 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:54.688085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:54.689480 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:54.691297 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:54.691304 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:54.691305 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:54.691480 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:54.691688 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:54.691689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:54.691689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:54.691739 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:54.692092 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:54.692098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:54.692135 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:50:54.692196 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:50:54.692281 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":81,"threads_started":1}
I20260812 03:50:54.692454 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.684112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.7.132.129:45463)
0812 03:50:54.684262 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:54.684267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:54.684449 (+ 182us) server_negotiation.cc:407] Connection header received
0812 03:50:54.684548 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:54.684550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:54.684563 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:54.684638 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:54.685186 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.686086 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.687190 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.687498 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.687607 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.689558 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:54.689566 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:54.689566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:54.689573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:54.691223 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.691555 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.691557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.691558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.691616 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.691800 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.691801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.691802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.691981 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:54.692117 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:54.692259 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:50:54.692360 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260812 03:50:54.692972 726 tablet_service.cc:1514] Processing DeleteTablet for tablet e25ea77152d94b7eaf03539a45e10485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:34 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:46150
I20260812 03:50:54.693429 873 tablet_replica.cc:330] stopping tablet replica
I20260812 03:50:54.693527 873 raft_consensus.cc:2229] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:50:54.693585 873 raft_consensus.cc:2258] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:54.693861 873 ts_tablet_manager.cc:1905] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:54.696043 873 ts_tablet_manager.cc:1918] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:54.696111 873 log.cc:1196] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal/wals/e25ea77152d94b7eaf03539a45e10485
I20260812 03:50:54.696344 873 ts_tablet_manager.cc:1939] T e25ea77152d94b7eaf03539a45e10485 P 37ede9777b894344ba4330905bad13ce: Deleting consensus metadata
I20260812 03:50:54.696842 526 catalog_manager.cc:4841] TS 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463): tablet e25ea77152d94b7eaf03539a45e10485 (table test-table [id=1810df33822f435f84d2e86459492fa8]) successfully deleted
W20260812 03:50:54.759320 867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:54.759531 867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:54.759569 867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:54.760869 867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:54.760928 867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:54.760972 867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:50:54.762364 867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:34857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=45215
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:54.762615 867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:54.762882 867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:54.767510 888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:54.767535 882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:54.767586 891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:54.767714 867 server_base.cc:1034] running on GCE node
I20260812 03:50:54.767942 867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:54.768181 867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:54.769340 867 hybrid_clock.cc:648] HybridClock initialized: now 1786506654769321 us; error 36 us; skew 500 ppm
Aug 12 03:50:54 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506654, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:54.771682 867 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:50:54.772125 867 webserver.cc:460] Webserver started at http://127.7.132.130:45215/ using document root <none> and password file <none>
I20260812 03:50:54.772354 867 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:54.772434 867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:54.773656 867 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:54.774185 898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.774351 867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:50:54.774448 867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "58e444ce901f4511b9558675d03aed8a"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "ad93e1e9502c435ff111f88408ea0674"
server_key_iv: "21a60575ee09e84f62b228346bae6896"
server_key_version: "encryptionkey@0"
I20260812 03:50:54.774803 867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:54.791396 867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:54.793120 867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:54.793289 867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:54.793486 867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:54.793766 867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:54.793816 867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.793856 867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:54.793884 867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.799662 867 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:34857
I20260812 03:50:54.799726 1079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:34857 every 8 connection(s)
I20260812 03:50:54.800029 867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:50:54.800493 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 867
I20260812 03:50:54.800818 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:40681
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:43005
--webserver_port=42775
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:50:54 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506654, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:54.810652 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.800682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:39367 (local address 127.7.132.190:43005)
0812 03:50:54.800732 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:54.800734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:54.801001 (+ 267us) server_negotiation.cc:407] Connection header received
0812 03:50:54.801802 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:54.801804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:54.801821 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:54.801873 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:54.803097 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.803858 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.805332 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.805533 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.805587 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.807955 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:54.807957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:54.807957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:54.807964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:54.809321 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.809720 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.809722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.809722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.809788 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.810027 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.810028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.810029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.810138 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:54.810211 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:54.810475 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:50:54.810566 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:50:54.810837 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.800673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:39367)
0812 03:50:54.800866 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:54.800913 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:54.801649 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:54.801970 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:54.801974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:54.802795 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:54.803021 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.803026 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.803956 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.803956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.805245 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.805248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.805625 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.805626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.805779 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.806255 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:54.806266 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:54.807857 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:54.809420 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:54.809427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:54.809430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:54.809643 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:54.809854 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:54.809855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:54.809855 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:54.809960 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:54.810203 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:54.810210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:54.810348 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 03:50:54.810452 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:50:54.810595 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260812 03:50:54.811372 1080 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43005
I20260812 03:50:54.811479 1080 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:54.811682 1080 heartbeater.cc:511] Master 127.7.132.190:43005 requested a full tablet report, sending...
I20260812 03:50:54.812078 551 ts_manager.cc:194] Registered new tserver with Master: 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857)
I20260812 03:50:54.812430 551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:39367
W20260812 03:50:54.884194 1085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:50:54.884492 1085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:50:54.884532 1085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:50:54.885687 1085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:50:54.885746 1085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:50:54.885788 1085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:50:54.887316 1085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:40681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=42775
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:50:54.887569 1085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:50:54.887786 1085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:50:54.892342 1092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:54.892343 1098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:50:54.892366 1093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:50:54.892577 1085 server_base.cc:1034] running on GCE node
I20260812 03:50:54.892778 1085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:50:54.893014 1085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:50:54.894166 1085 hybrid_clock.cc:648] HybridClock initialized: now 1786506654894148 us; error 49 us; skew 500 ppm
Aug 12 03:50:54 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506654, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:50:54.896441 1085 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:50:54.896911 1085 webserver.cc:460] Webserver started at http://127.7.132.131:42775/ using document root <none> and password file <none>
I20260812 03:50:54.897149 1085 fs_manager.cc:359] Metadata directory not provided
I20260812 03:50:54.897233 1085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:50:54.898491 1085 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:54.899155 1106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.899340 1085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.899436 1085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "8daa756e125c4a758a2d0dc573c8317e"
format_stamp: "Formatted at 2026-08-12 03:50:34 on dist-test-slave-39l8"
server_key: "52f6a40660ec734179cc5dddb247ad05"
server_key_iv: "955ae6b51a6f1b84d576d69f062f7a6b"
server_key_version: "encryptionkey@0"
I20260812 03:50:54.899757 1085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:50:54.914870 1085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:50:54.916661 1085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:50:54.916817 1085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:50:54.917049 1085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:50:54.917359 1085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:50:54.917410 1085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.917449 1085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:50:54.917474 1085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.923398 1085 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:40681
I20260812 03:50:54.923440 1290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:40681 every 8 connection(s)
I20260812 03:50:54.923810 1085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:50:54.927387 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 1085
Aug 12 03:50:54 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506654, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:50:54.934237 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.924535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:34939 (local address 127.7.132.190:43005)
0812 03:50:54.924619 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:54.924622 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:54.924798 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:50:54.925336 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:54.925338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:54.925351 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:54.925397 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:54.926353 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.926915 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.928234 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.928552 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.928634 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.931025 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:54.931026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:54.931027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:54.931032 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:54.932806 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.933276 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.933277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.933278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.933336 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.933578 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.933579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.933579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.933751 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:54.933823 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:54.934113 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 03:50:54.934174 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:50:54.934501 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.924493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:34939)
0812 03:50:54.924672 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:54.924715 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:54.925220 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:54.925477 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:54.925481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:54.926025 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:50:54.926247 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.926255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.927009 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.927010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.928127 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.928131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.928638 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.928639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.928745 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.929228 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:50:54.929238 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:50:54.930926 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:50:54.932892 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:54.932898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:54.932899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:54.933197 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:54.933412 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:50:54.933413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:50:54.933413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:50:54.933509 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:50:54.933819 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:50:54.933823 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:50:54.933952 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 03:50:54.934087 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 03:50:54.934265 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260812 03:50:54.935083 1295 heartbeater.cc:347] Connected to a master server at 127.7.132.190:43005
I20260812 03:50:54.935127 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.927827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33436 (local address 127.7.132.190:43005)
0812 03:50:54.927962 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:54.927964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:54.928043 (+ 79us) server_negotiation.cc:407] Connection header received
0812 03:50:54.928075 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:54.928076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:54.928087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:54.928156 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:54.928874 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.929522 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.930516 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.930762 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.930820 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.932286 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:54.932287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:54.932287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:54.932305 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:54.933918 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.934293 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.934294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.934294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.934351 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:54.934559 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:54.934560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:54.934560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:54.934683 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:54.934896 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:54.934936 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 03:50:54.935015 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260812 03:50:54.935204 1295 heartbeater.cc:464] Registering TS with master...
I20260812 03:50:54.935415 1295 heartbeater.cc:511] Master 127.7.132.190:43005 requested a full tablet report, sending...
I20260812 03:50:54.935840 549 ts_manager.cc:194] Registered new tserver with Master: 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
I20260812 03:50:54.936210 549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:34939
I20260812 03:50:54.946936 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:50:54.949724 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.947362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.7.132.190:43005)
0812 03:50:54.947436 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:54.947437 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:54.947547 (+ 110us) server_negotiation.cc:407] Connection header received
0812 03:50:54.947598 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:54.947598 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:54.947628 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:54.947680 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:50:54.947793 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.948235 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.948960 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.949131 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.949202 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.949395 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:50:54.949481 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:50:54.949559 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:54.949616 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 03:50:54.949640 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:50:54.949991 549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:50:54.951727 549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:50:54.953840 726 tablet_service.cc:1467] Processing CreateTablet for tablet 70d01a7184dd46a6bb2e3e3a267b13d4 (DEFAULT_TABLE table=test-table [id=9b753cacfe4a4b8ba7acfe6ba9384762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:54.954041 726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d01a7184dd46a6bb2e3e3a267b13d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:54.955576 685 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Bootstrap starting.
I20260812 03:50:54.955909 685 tablet_bootstrap.cc:654] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:54.956743 685 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: No bootstrap required, opened a new log
I20260812 03:50:54.956813 685 ts_tablet_manager.cc:1397] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:50:54.956966 685 raft_consensus.cc:348] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:54.957043 685 raft_consensus.cc:374] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:54.957098 685 raft_consensus.cc:729] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ede9777b894344ba4330905bad13ce, State: Initialized, Role: FOLLOWER
I20260812 03:50:54.957159 685 consensus_queue.cc:260] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:54.957259 685 ts_tablet_manager.cc:1428] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 03:50:54.957597 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.953434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:40681 (local address 127.0.0.1:60988)
0812 03:50:54.953565 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:54.953575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:54.953618 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:54.954150 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:54.954150 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:54.954152 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:54.954184 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.954185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.955192 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.955193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.956412 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.956415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.957342 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.957343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.957416 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.957436 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:50:54.957495 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:50:54.957534 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":29,"thread_start_us":83,"threads_started":1}
I20260812 03:50:54.957638 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.953830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.7.132.131:40681)
0812 03:50:54.953969 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:54.953973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:54.953982 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:50:54.954016 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:54.954018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:54.954032 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:54.954151 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:54.954226 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.955108 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.956516 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.957193 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.957288 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.957347 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:54.957516 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 03:50:54.957562 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260812 03:50:54.958397 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.953412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34857 (local address 127.0.0.1:44372)
0812 03:50:54.953456 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:54.953466 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:54.953499 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:54.954021 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:54.954022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:54.954023 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:54.954066 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.954068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.955245 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.955246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.956412 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:54.956415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.958181 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.958181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:54.958246 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.958250 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:50:54.958303 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:50:54.958340 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":35}
I20260812 03:50:54.958318 1191 tablet_service.cc:1467] Processing CreateTablet for tablet 70d01a7184dd46a6bb2e3e3a267b13d4 (DEFAULT_TABLE table=test-table [id=9b753cacfe4a4b8ba7acfe6ba9384762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:54.958424 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:54.953659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.7.132.130:34857)
0812 03:50:54.953860 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:54.953865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:54.953876 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:50:54.953913 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:54.953915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:54.953930 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:54.954015 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:54.954138 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.955173 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.957392 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:54.958099 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:54.958166 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:54.958215 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:54.958317 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 03:50:54.958353 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":102,"threads_started":1}
I20260812 03:50:54.958674 1191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d01a7184dd46a6bb2e3e3a267b13d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:54.959112 977 tablet_service.cc:1467] Processing CreateTablet for tablet 70d01a7184dd46a6bb2e3e3a267b13d4 (DEFAULT_TABLE table=test-table [id=9b753cacfe4a4b8ba7acfe6ba9384762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:50:54.959410 977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d01a7184dd46a6bb2e3e3a267b13d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:50:54.961480 1312 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Bootstrap starting.
I20260812 03:50:54.962105 1313 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Bootstrap starting.
I20260812 03:50:54.962159 1312 tablet_bootstrap.cc:654] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:54.962635 1312 log.cc:824] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:54.962787 1313 tablet_bootstrap.cc:654] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Neither blocks nor log segments found. Creating new log.
I20260812 03:50:54.963384 1313 log.cc:824] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:50:54.963423 1312 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: No bootstrap required, opened a new log
I20260812 03:50:54.963505 1312 ts_tablet_manager.cc:1397] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:50:54.964090 1313 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: No bootstrap required, opened a new log
I20260812 03:50:54.964171 1313 ts_tablet_manager.cc:1397] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:50:54.965014 1312 raft_consensus.cc:348] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:54.965226 1312 raft_consensus.cc:374] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:54.965317 1312 raft_consensus.cc:729] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8daa756e125c4a758a2d0dc573c8317e, State: Initialized, Role: FOLLOWER
I20260812 03:50:54.965487 1312 consensus_queue.cc:260] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:54.965772 1312 ts_tablet_manager.cc:1428] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:50:54.965801 1295 heartbeater.cc:503] Master 127.7.132.190:43005 was elected leader, sending a full tablet report...
I20260812 03:50:54.965770 1313 raft_consensus.cc:348] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:54.965925 1313 raft_consensus.cc:374] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:50:54.965955 1313 raft_consensus.cc:729] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e444ce901f4511b9558675d03aed8a, State: Initialized, Role: FOLLOWER
I20260812 03:50:54.966073 1313 consensus_queue.cc:260] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:54.966369 1313 ts_tablet_manager.cc:1428] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:50:54.966391 1080 heartbeater.cc:503] Master 127.7.132.190:43005 was elected leader, sending a full tablet report...
W20260812 03:50:55.052994 1081 tablet.cc:2369] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:50:55.169646 861 tablet.cc:2369] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:50:55.174482 1298 tablet.cc:2369] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:50:55.188706 1320 raft_consensus.cc:482] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:50:55.188849 1320 raft_consensus.cc:504] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:55.189239 1320 leader_election.cc:288] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857), 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463)
I20260812 03:50:55.192059 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:55.189434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:38365 (local address 127.7.132.130:34857)
0812 03:50:55.189475 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:55.189477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:55.189537 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:50:55.189620 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:55.189622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:55.189631 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:55.189730 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:55.189906 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.190397 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.191371 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.191723 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.191806 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:55.191815 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:55.191952 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:50:55.191990 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 03:50:55.192059 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:55.189402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34857 (local address 127.7.132.131:38365)
0812 03:50:55.189470 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:55.189481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:55.189535 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:55.189742 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:55.189743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:55.189746 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:55.189818 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:55.189820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.190487 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.190489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:55.191281 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:55.191284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.191803 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.191804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:55.191880 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:55.191883 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:50:55.191945 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:50:55.191981 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 03:50:55.192251 1322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:55.189431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45463 (local address 127.7.132.131:49287)
0812 03:50:55.189696 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:55.189704 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:50:55.189762 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:50:55.189941 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:50:55.189942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:50:55.189944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:50:55.190015 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:55.190017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.190664 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.190665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:55.191453 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:50:55.191456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.191987 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.191988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:50:55.192082 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:55.192087 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:50:55.192136 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:50:55.192176 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":38,"thread_start_us":119,"threads_started":1}
I20260812 03:50:55.192252 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:55.189513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:49287 (local address 127.7.132.129:45463)
0812 03:50:55.189574 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:55.189576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:55.189762 (+ 186us) server_negotiation.cc:407] Connection header received
0812 03:50:55.189815 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:55.189816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:55.189843 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:55.189921 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:50:55.190082 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.190568 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.191539 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.191903 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.191985 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:55.192079 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:55.192160 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 03:50:55.192198 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 03:50:55.192660 1004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d01a7184dd46a6bb2e3e3a267b13d4" candidate_uuid: "8daa756e125c4a758a2d0dc573c8317e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58e444ce901f4511b9558675d03aed8a" is_pre_election: true
I20260812 03:50:55.192647 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d01a7184dd46a6bb2e3e3a267b13d4" candidate_uuid: "8daa756e125c4a758a2d0dc573c8317e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ede9777b894344ba4330905bad13ce" is_pre_election: true
I20260812 03:50:55.192803 1004 raft_consensus.cc:2454] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8daa756e125c4a758a2d0dc573c8317e in term 0.
I20260812 03:50:55.192802 789 raft_consensus.cc:2454] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8daa756e125c4a758a2d0dc573c8317e in term 0.
I20260812 03:50:55.193035 1107 leader_election.cc:302] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37ede9777b894344ba4330905bad13ce, 8daa756e125c4a758a2d0dc573c8317e; no voters:
I20260812 03:50:55.193145 1320 raft_consensus.cc:2781] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:50:55.193200 1320 raft_consensus.cc:482] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:50:55.193233 1320 raft_consensus.cc:3037] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:55.193931 1320 raft_consensus.cc:504] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:55.194056 1320 leader_election.cc:288] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [CANDIDATE]: Term 1 election: Requested vote from peers 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857), 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463)
I20260812 03:50:55.194226 1004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d01a7184dd46a6bb2e3e3a267b13d4" candidate_uuid: "8daa756e125c4a758a2d0dc573c8317e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58e444ce901f4511b9558675d03aed8a"
I20260812 03:50:55.194226 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d01a7184dd46a6bb2e3e3a267b13d4" candidate_uuid: "8daa756e125c4a758a2d0dc573c8317e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ede9777b894344ba4330905bad13ce"
I20260812 03:50:55.194304 789 raft_consensus.cc:3037] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:55.194305 1004 raft_consensus.cc:3037] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:50:55.195123 789 raft_consensus.cc:2454] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8daa756e125c4a758a2d0dc573c8317e in term 1.
I20260812 03:50:55.195170 1004 raft_consensus.cc:2454] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8daa756e125c4a758a2d0dc573c8317e in term 1.
I20260812 03:50:55.195284 1107 leader_election.cc:302] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37ede9777b894344ba4330905bad13ce, 8daa756e125c4a758a2d0dc573c8317e; no voters:
I20260812 03:50:55.195384 1320 raft_consensus.cc:2781] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:50:55.195510 1320 raft_consensus.cc:686] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 1 LEADER]: Becoming Leader. State: Replica: 8daa756e125c4a758a2d0dc573c8317e, State: Running, Role: LEADER
I20260812 03:50:55.195597 1320 consensus_queue.cc:237] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:50:55.196368 551 catalog_manager.cc:5495] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131). New cstate: current_term: 1 leader_uuid: "8daa756e125c4a758a2d0dc573c8317e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } health_report { overall_health: UNKNOWN } } }
I20260812 03:50:55.239279 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:55.236973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.7.132.131:40681)
0812 03:50:55.237036 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:55.237038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:55.237048 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:50:55.237087 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:55.237088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:55.237106 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:55.237194 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:50:55.237337 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.237802 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.238511 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.238760 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.238821 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:55.238984 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:50:55.239036 (+ 52us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 03:50:55.239188 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32770: token expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 03:50:55.239344 1310 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32770: token expired
I20260812 03:50:55.247781 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:55.239621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33446 (local address 127.7.132.190:43005)
0812 03:50:55.239661 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:55.239662 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:55.239670 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:50:55.239704 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:55.239705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:55.239712 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:55.239754 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:50:55.240183 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.240628 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.241266 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.241476 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.241523 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:55.245512 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:50:55.245513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:50:55.245514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:50:55.245519 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:50:55.246888 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:55.247177 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:55.247178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:55.247178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:55.247229 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:50:55.247386 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:50:55.247387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:50:55.247387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:50:55.247490 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:50:55.247561 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:55.247661 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 03:50:55.247721 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:50:55.250747 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:55.248589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32778 (local address 127.7.132.131:40681)
0812 03:50:55.248646 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:50:55.248654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:50:55.248662 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:50:55.248704 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:50:55.248704 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:50:55.248714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:50:55.248766 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:50:55.248898 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.249332 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.250065 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:50:55.250250 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:50:55.250315 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:50:55.250435 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:50:55.250517 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:50:55.250570 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:50:55.250635 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:50:55.250660 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:50:55.252453 789 raft_consensus.cc:1261] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Refusing update from remote peer 8daa756e125c4a758a2d0dc573c8317e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:55.252453 1004 raft_consensus.cc:1261] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 1 FOLLOWER]: Refusing update from remote peer 8daa756e125c4a758a2d0dc573c8317e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:50:55.252837 1323 consensus_queue.cc:1035] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:55.252975 1320 consensus_queue.cc:1035] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:50:55.254825 1327 mvcc.cc:205] Tried to move back new op lower bound from 7317531259911233536 to 7317531259681726464. Current Snapshot: MvccSnapshot[applied={T|T < 7317531259911233536}]
I20260812 03:50:55.255654 1333 mvcc.cc:205] Tried to move back new op lower bound from 7317531259911233536 to 7317531259681726464. Current Snapshot: MvccSnapshot[applied={T|T < 7317531259911233536}]
I20260812 03:50:55.255455 1331 mvcc.cc:205] Tried to move back new op lower bound from 7317531259911233536 to 7317531259681726464. Current Snapshot: MvccSnapshot[applied={T|T < 7317531259911233536 or (T in {7317531259911233536})}]
I20260812 03:50:55.257484 549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:55.257575 549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:50:55.258244 549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Sending DeleteTablet for 3 replicas of tablet 70d01a7184dd46a6bb2e3e3a267b13d4
I20260812 03:50:55.258531 977 tablet_service.cc:1514] Processing DeleteTablet for tablet 70d01a7184dd46a6bb2e3e3a267b13d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:55 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:44372
I20260812 03:50:55.258572 1191 tablet_service.cc:1514] Processing DeleteTablet for tablet 70d01a7184dd46a6bb2e3e3a267b13d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:55 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:60988
I20260812 03:50:55.258599 726 tablet_service.cc:1514] Processing DeleteTablet for tablet 70d01a7184dd46a6bb2e3e3a267b13d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:50:55 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:46150
I20260812 03:50:55.258829 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 614
I20260812 03:50:55.258852 1344 tablet_replica.cc:330] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: stopping tablet replica
I20260812 03:50:55.265938 1344 raft_consensus.cc:2229] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:50:55.266077 1344 raft_consensus.cc:2258] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:55.267262 1344 ts_tablet_manager.cc:1905] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:55.268978 1344 ts_tablet_manager.cc:1918] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:55.269048 1344 log.cc:1196] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/70d01a7184dd46a6bb2e3e3a267b13d4
I20260812 03:50:55.269289 1344 ts_tablet_manager.cc:1939] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 58e444ce901f4511b9558675d03aed8a: Deleting consensus metadata
I20260812 03:50:55.269616 1345 tablet_replica.cc:330] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: stopping tablet replica
I20260812 03:50:55.269793 1345 raft_consensus.cc:2229] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:50:55.269776 531 catalog_manager.cc:4841] TS 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857): tablet 70d01a7184dd46a6bb2e3e3a267b13d4 (table test-table [id=9b753cacfe4a4b8ba7acfe6ba9384762]) successfully deleted
I20260812 03:50:55.269963 1345 raft_consensus.cc:2258] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:50:55.270632 1345 ts_tablet_manager.cc:1905] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:50:55.271811 1345 ts_tablet_manager.cc:1918] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:50:55.271891 1345 log.cc:1196] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/70d01a7184dd46a6bb2e3e3a267b13d4
I20260812 03:50:55.272230 1345 ts_tablet_manager.cc:1939] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 8daa756e125c4a758a2d0dc573c8317e: Deleting consensus metadata
I20260812 03:50:55.272595 526 catalog_manager.cc:4841] TS 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681): tablet 70d01a7184dd46a6bb2e3e3a267b13d4 (table test-table [id=9b753cacfe4a4b8ba7acfe6ba9384762]) successfully deleted
W20260812 03:50:55.281991 526 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:45463 (error 108)
I20260812 03:50:55.282238 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 867
I20260812 03:50:55.284566 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:50:55.282409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45463 (local address 127.0.0.1:46156)
0812 03:50:55.284487 (+ 2078us) negotiation.cc:107] Waiting for socket to connect
0812 03:50:55.284530 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2054}
W20260812 03:50:55.287611 526 catalog_manager.cc:4584] TS 37ede9777b894344ba4330905bad13ce (127.7.132.129:45463): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70d01a7184dd46a6bb2e3e3a267b13d4: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45463: connect: Connection refused (error 111)
I20260812 03:50:55.299710 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 1085
I20260812 03:50:55.308475 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 503
I20260812 03:51:15.318994 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43005
--webserver_interface=127.7.132.190
--webserver_port=46489
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:43005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:15.397686 1350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:15.397907 1350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:15.397941 1350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:15.399091 1350 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:51:15.399153 1350 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:51:15.399174 1350 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:51:15.399194 1350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:15.399246 1350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:15.399271 1350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:15.399295 1350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:15.399318 1350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:15.400717 1350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:43005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:43005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=46489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:15.401021 1350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:15.401244 1350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:15.406075 1368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:15.406067 1361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:15.406080 1360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:15.406406 1350 server_base.cc:1034] running on GCE node
I20260812 03:51:15.406634 1350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:15.406942 1350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:15.408084 1350 hybrid_clock.cc:648] HybridClock initialized: now 1786506675408071 us; error 30 us; skew 500 ppm
Aug 12 03:51:15 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506675, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:15.415004 1350 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:51:15.415428 1350 webserver.cc:460] Webserver started at http://127.7.132.190:46489/ using document root <none> and password file <none>
I20260812 03:51:15.415665 1350 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:15.415710 1350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:15.417009 1350 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:15.417639 1378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.417807 1350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.417882 1350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "6ace52b60fdc4485be60a510b4a93f40"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "3d7bbde7e389f2dec0717b0388958476"
server_key_iv: "f2fb9866e7afb257b27daa7923537b58"
server_key_version: "encryptionkey@0"
I20260812 03:51:15.418210 1350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:15.426479 1350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:15.428207 1350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:15.428313 1350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:15.432240 1350 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:43005
I20260812 03:51:15.432291 1460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:43005 every 8 connection(s)
I20260812 03:51:15.432638 1350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:15.434031 1463 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:51:15.434237 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 1350
I20260812 03:51:15.434511 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:45463
--local_ip_for_outbound_sockets=127.7.132.129
--tserver_master_addrs=127.7.132.190:43005
--webserver_port=40805
--webserver_interface=127.7.132.129
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:51:15.434851 1463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Bootstrap starting.
I20260812 03:51:15.444101 1463 log.cc:824] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:15.446141 1463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:51:15.446470 1463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Bootstrap complete.
I20260812 03:51:15.448794 1463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:51:15.449100 1463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ace52b60fdc4485be60a510b4a93f40, State: Initialized, Role: FOLLOWER
I20260812 03:51:15.449287 1463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:51:15.449360 1463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:15.449412 1463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:15.449455 1463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:51:15.450338 1463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:51:15.450448 1463 leader_election.cc:302] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ace52b60fdc4485be60a510b4a93f40; no voters:
I20260812 03:51:15.450609 1463 leader_election.cc:288] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:51:15.450657 1468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:51:15.450881 1463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:15.450908 1468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [term 3 LEADER]: Becoming Leader. State: Replica: 6ace52b60fdc4485be60a510b4a93f40, State: Running, Role: LEADER
I20260812 03:51:15.450994 1468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } }
I20260812 03:51:15.451287 1468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ace52b60fdc4485be60a510b4a93f40. Latest consensus state: current_term: 3 leader_uuid: "6ace52b60fdc4485be60a510b4a93f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } } }
I20260812 03:51:15.451409 1468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:15.451370 1469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6ace52b60fdc4485be60a510b4a93f40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ace52b60fdc4485be60a510b4a93f40" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 43005 } } }
I20260812 03:51:15.451542 1469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:15.451929 1483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:15.452800 1483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:15.452904 1483 catalog_manager.cc:1259] Loaded cluster ID: 4cf188fc27fa4f458fc6aff7e19a75c1
I20260812 03:51:15.452939 1483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:15.454105 1483 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:15.454273 1483 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Loaded TSK: 0
I20260812 03:51:15.454312 1483 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ace52b60fdc4485be60a510b4a93f40: Loaded TSK: 1
F20260812 03:51:15.454547 1483 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f97effda9ad google::LogMessage::Fail() at ??:0
@ 0x7f97effdd88a google::LogMessage::SendToLog() at ??:0
@ 0x7f97effda4de google::LogMessage::Flush() at ??:0
@ 0x7f97effdba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f97f2747104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f97f2744d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f97f7f341cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f97f7f3448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f97f7f34594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f97f7efac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f97f7f04ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f97f1ea80d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f97f1e9d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f97f2d016db start_thread at ??:0
@ 0x7f97eead861f clone at ??:0
W20260812 03:51:15.529500 1465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:15.529680 1465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:15.529699 1465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:15.530776 1465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:15.530814 1465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:15.530838 1465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:51:15.532243 1465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:45463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=40805
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:15.532457 1465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:15.532694 1465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:15.537530 1530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:15.537632 1465 server_base.cc:1034] running on GCE node
W20260812 03:51:15.537652 1533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:15.537556 1531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:15.537901 1465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:15.538183 1465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:15.539332 1465 hybrid_clock.cc:648] HybridClock initialized: now 1786506675539314 us; error 36 us; skew 500 ppm
Aug 12 03:51:15 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506675, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:15.541747 1465 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:51:15.542191 1465 webserver.cc:460] Webserver started at http://127.7.132.129:40805/ using document root <none> and password file <none>
I20260812 03:51:15.542399 1465 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:15.542439 1465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:15.543716 1465 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:15.544392 1543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.544579 1465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:15.544663 1465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "37ede9777b894344ba4330905bad13ce"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "8d2e1f18641cd1b08ed053f1b3178530"
server_key_iv: "8312ebe594b42b7c402d3ec36da4a14e"
server_key_version: "encryptionkey@0"
I20260812 03:51:15.544979 1465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:15.558871 1465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:15.560850 1465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:15.560981 1465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:15.561185 1465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:15.561565 1553 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:51:15.563562 1465 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:51:15.563619 1465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 03:51:15.563663 1465 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:51:15.564242 1465 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:51:15.564296 1465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 03:51:15.564361 1553 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Bootstrap starting.
I20260812 03:51:15.571309 1465 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45463
I20260812 03:51:15.571390 1721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45463 every 8 connection(s)
I20260812 03:51:15.571713 1465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:51:15.575505 1553 log.cc:824] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:15.578233 1553 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:51:15.578567 1553 tablet_bootstrap.cc:492] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Bootstrap complete.
I20260812 03:51:15.579890 1553 ts_tablet_manager.cc:1397] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260812 03:51:15.580631 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 1465
I20260812 03:51:15.580901 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:34857
--local_ip_for_outbound_sockets=127.7.132.130
--tserver_master_addrs=127.7.132.190:43005
--webserver_port=45215
--webserver_interface=127.7.132.130
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:51:15.581326 1553 raft_consensus.cc:348] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:51:15.581657 1553 raft_consensus.cc:729] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ede9777b894344ba4330905bad13ce, State: Initialized, Role: FOLLOWER
I20260812 03:51:15.581846 1553 consensus_queue.cc:260] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:51:15.582119 1553 ts_tablet_manager.cc:1428] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260812 03:51:15.653105 1725 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:51:15.572399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:46541)
0812 03:51:15.572565 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.572605 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:15.573011 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:15.652955 (+ 79944us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
W20260812 03:51:15.653345 1723 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:43005 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:43005: Client connection negotiation failed: client connection to 127.7.132.190:43005: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:51:15.653682 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:15.653561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:53945)
0812 03:51:15.653616 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.653655 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 03:51:15.654076 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:15.653997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:35117)
0812 03:51:15.654039 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.654061 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:51:15.654201 1723 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:51:15.667819 1729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:15.668004 1729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:15.668027 1729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:15.669064 1729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:15.669099 1729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:15.669131 1729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:51:15.670495 1729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:34857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=45215
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:15.670670 1729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:15.670898 1729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:15.675647 1736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:15.675652 1734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:15.675725 1729 server_base.cc:1034] running on GCE node
W20260812 03:51:15.675652 1738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:15.676024 1729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:15.676254 1729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:15.677410 1729 hybrid_clock.cc:648] HybridClock initialized: now 1786506675677407 us; error 35 us; skew 500 ppm
Aug 12 03:51:15 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506675, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:15.679755 1729 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:51:15.680189 1729 webserver.cc:460] Webserver started at http://127.7.132.130:45215/ using document root <none> and password file <none>
I20260812 03:51:15.680419 1729 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:15.680487 1729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:15.681784 1729 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:15.682367 1748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.682535 1729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:15.682638 1729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "58e444ce901f4511b9558675d03aed8a"
format_stamp: "Formatted at 2026-08-12 03:50:33 on dist-test-slave-39l8"
server_key: "ad93e1e9502c435ff111f88408ea0674"
server_key_iv: "21a60575ee09e84f62b228346bae6896"
server_key_version: "encryptionkey@0"
I20260812 03:51:15.683008 1729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:15.705368 1729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:15.707228 1729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:15.707393 1729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:15.707628 1729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:15.707952 1729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:15.708015 1729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.708076 1729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:15.708103 1729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.714078 1729 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:34857
I20260812 03:51:15.714128 1933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:34857 every 8 connection(s)
I20260812 03:51:15.714458 1729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:51:15.715477 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:15.715109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:34349)
0812 03:51:15.715275 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.715344 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
W20260812 03:51:15.715888 1934 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:43005 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:43005: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
I20260812 03:51:15.716176 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:15.716082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:46143)
0812 03:51:15.716128 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.716154 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:51:15.716456 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:15.716374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:36277)
0812 03:51:15.716415 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.716436 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 03:51:15.716563 1934 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:51:15.717512 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 1729
I20260812 03:51:15.717780 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:40681
--local_ip_for_outbound_sockets=127.7.132.131
--tserver_master_addrs=127.7.132.190:43005
--webserver_port=42775
--webserver_interface=127.7.132.131
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:15.799513 1938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:15.799719 1938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:15.799753 1938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:15.800827 1938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:15.800891 1938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:15.800956 1938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:51:15.802434 1938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:40681
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=42775
--tserver_master_addrs=127.7.132.190:43005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:15.802681 1938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:15.802923 1938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:15.807693 1944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:15.807714 1948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:15.807726 1945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:15.807848 1938 server_base.cc:1034] running on GCE node
I20260812 03:51:15.808087 1938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:15.808359 1938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:15.809501 1938 hybrid_clock.cc:648] HybridClock initialized: now 1786506675809490 us; error 29 us; skew 500 ppm
Aug 12 03:51:15 dist-test-slave-39l8 krb5kdc[32085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506675, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:15.811976 1938 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:51:15.812493 1938 webserver.cc:460] Webserver started at http://127.7.132.131:42775/ using document root <none> and password file <none>
I20260812 03:51:15.812739 1938 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:15.812811 1938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:15.814075 1938 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:51:15.814606 1956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.814805 1938 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:15.814898 1938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "8daa756e125c4a758a2d0dc573c8317e"
format_stamp: "Formatted at 2026-08-12 03:50:34 on dist-test-slave-39l8"
server_key: "52f6a40660ec734179cc5dddb247ad05"
server_key_iv: "955ae6b51a6f1b84d576d69f062f7a6b"
server_key_version: "encryptionkey@0"
I20260812 03:51:15.815227 1938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:15.830679 1938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:15.833033 1938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:15.833194 1938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:15.833410 1938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:15.833707 1938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:15.833762 1938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.833801 1938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:15.833829 1938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:15.839563 1938 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:40681
I20260812 03:51:15.839613 2139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:40681 every 8 connection(s)
I20260812 03:51:15.839918 1938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:51:15.840884 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:15.840531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:51051)
0812 03:51:15.840673 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.840745 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260812 03:51:15.841369 2140 heartbeater.cc:644] Failed to heartbeat to 127.7.132.190:43005 (0 consecutive failures): Network error: Failed to ping master at 127.7.132.190:43005: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
I20260812 03:51:15.841642 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:15.841543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:51427)
0812 03:51:15.841588 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.841613 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:51:15.841868 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:15.841823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:37923)
0812 03:51:15.841842 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:15.841858 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260812 03:51:15.841989 2140 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:51:15.843307 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 1938
I20260812 03:51:16.654893 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:16.654510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:47565)
0812 03:51:16.654647 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:16.654680 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":58,"threads_started":1}
I20260812 03:51:16.717180 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:16.716825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:34509)
0812 03:51:16.717027 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:16.717061 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":52,"threads_started":1}
I20260812 03:51:16.842470 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:16.842275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:42707)
0812 03:51:16.842376 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:16.842425 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
I20260812 03:51:17.022328 2156 raft_consensus.cc:482] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:17.022495 2156 raft_consensus.cc:504] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:51:17.022876 2156 leader_election.cc:288] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857), 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
Aug 12 03:51:17 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506675, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.130@KRBTEST.COM
Aug 12 03:51:17 dist-test-slave-39l8 krb5kdc[32085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506675, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.131@KRBTEST.COM
I20260812 03:51:17.035171 2161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:17.023189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:46755 (local address 127.7.132.130:34857)
0812 03:51:17.023327 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:17.023339 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:17.023372 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:51:17.024001 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:17.024004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:17.024065 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:17.024220 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:17.025281 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:17.026614 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:17.028099 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:17.028750 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:17.028859 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:17.031562 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:17.031573 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:17.031573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:17.031583 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:17.033430 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:17.033881 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:17.033884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:17.033887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:17.033974 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:17.034241 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:17.034242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:17.034242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:17.034468 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:17.034584 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:17.034937 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 03:51:17.035056 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260812 03:51:17.035167 2149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:17.023049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:34857 (local address 127.7.132.129:46755)
0812 03:51:17.023117 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:17.023132 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:17.023205 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:17.024297 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:17.024300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:17.024848 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:17.025140 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:17.025156 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:17.026756 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:17.026757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:17.028004 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:17.028007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:17.028853 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:17.028854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:17.029009 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:17.029648 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:17.029651 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:17.031446 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:17.033538 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:17.033546 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:17.033548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:17.033791 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:17.034051 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:17.034052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:17.034053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:17.034175 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:17.034565 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:17.034573 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:17.034750 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 03:51:17.034883 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 03:51:17.035069 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":44}
I20260812 03:51:17.035207 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:17.023221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:44901 (local address 127.7.132.131:40681)
0812 03:51:17.023379 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:17.023387 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:17.023414 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:51:17.024000 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:17.024003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:17.024065 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:17.024220 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:17.025281 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:17.026633 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:17.028101 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:17.028745 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:17.028859 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:17.031875 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:17.031878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:17.031878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:17.031888 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:17.033496 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:17.033902 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:17.033905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:17.033906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:17.033994 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:17.034251 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:17.034252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:17.034252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:17.034468 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:17.034584 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:17.034953 (+ 369us) server_negotiation.cc:299] Negotiation successful
0812 03:51:17.035088 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":55,"threads_started":1}
I20260812 03:51:17.035279 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:17.023049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:40681 (local address 127.7.132.129:44901)
0812 03:51:17.023233 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:17.023244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:17.023300 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:17.024297 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:17.024300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:17.024848 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:17.025140 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:17.025147 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:17.026747 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:17.026748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:17.028004 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:17.028007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:17.028851 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:17.028852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:17.029009 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:17.029636 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:17.029662 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:17.031761 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:17.033586 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:17.033587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:17.033588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:17.033821 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:17.034070 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:17.034071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:17.034071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:17.034182 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:17.034567 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:17.034574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:17.034752 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 03:51:17.034883 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 03:51:17.035150 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":271,"thread_start_us":82,"threads_started":1}
W20260812 03:51:17.036089 1544 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
W20260812 03:51:17.036585 1549 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
I20260812 03:51:17.036659 1549 leader_election.cc:302] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ede9777b894344ba4330905bad13ce; no voters: 58e444ce901f4511b9558675d03aed8a, 8daa756e125c4a758a2d0dc573c8317e
I20260812 03:51:17.036796 2156 raft_consensus.cc:2726] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:51:17.655552 2163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:17.655352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:52247)
0812 03:51:17.655470 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:17.655499 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260812 03:51:17.717795 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:17.717549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:40717)
0812 03:51:17.717715 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:17.717753 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260812 03:51:17.843230 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:17.843051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:55087)
0812 03:51:17.843162 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:17.843193 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260812 03:51:18.656199 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:18.655979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:60339)
0812 03:51:18.656120 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:18.656154 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260812 03:51:18.718413 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:18.718198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:56515)
0812 03:51:18.718327 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:18.718362 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260812 03:51:18.843873 2174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:18.843666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:59113)
0812 03:51:18.843792 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:18.843826 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260812 03:51:19.229031 2175 raft_consensus.cc:482] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:19.229130 2175 raft_consensus.cc:504] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:51:19.229301 2175 leader_election.cc:288] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857), 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
W20260812 03:51:19.229735 1544 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
W20260812 03:51:19.229818 1549 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
I20260812 03:51:19.229904 1549 leader_election.cc:302] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ede9777b894344ba4330905bad13ce; no voters: 58e444ce901f4511b9558675d03aed8a, 8daa756e125c4a758a2d0dc573c8317e
I20260812 03:51:19.229995 2175 raft_consensus.cc:2726] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:51:19.657047 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:19.656789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:58007)
0812 03:51:19.656953 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:19.656986 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260812 03:51:19.719040 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:19.718836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:46069)
0812 03:51:19.718960 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:19.718993 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260812 03:51:19.844445 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:19.844270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:48305)
0812 03:51:19.844381 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:19.844413 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260812 03:51:20.657894 2182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:20.657681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:48505)
0812 03:51:20.657818 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:20.657852 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260812 03:51:20.719636 2183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:20.719452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:50353)
0812 03:51:20.719566 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:20.719597 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260812 03:51:20.845119 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:20.844932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:33921)
0812 03:51:20.845057 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:20.845087 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260812 03:51:21.090051 2185 raft_consensus.cc:482] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:21.090186 2185 raft_consensus.cc:504] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:51:21.090401 2185 leader_election.cc:288] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857), 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
W20260812 03:51:21.090792 1544 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
W20260812 03:51:21.090873 1549 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
I20260812 03:51:21.090916 1549 leader_election.cc:302] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ede9777b894344ba4330905bad13ce; no voters: 58e444ce901f4511b9558675d03aed8a, 8daa756e125c4a758a2d0dc573c8317e
I20260812 03:51:21.090988 2185 raft_consensus.cc:2726] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:51:21.658530 2193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:21.658333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:42329)
0812 03:51:21.658449 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:21.658483 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260812 03:51:21.720221 2194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:21.720067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:49343)
0812 03:51:21.720164 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:21.720190 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":51,"threads_started":1}
I20260812 03:51:21.845703 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:21.845498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:49425)
0812 03:51:21.845628 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:21.845669 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260812 03:51:22.659192 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:22.658984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:34163)
0812 03:51:22.659108 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:22.659140 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260812 03:51:22.720839 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:22.720645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:46003)
0812 03:51:22.720768 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:22.720800 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260812 03:51:22.846352 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:22.846141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:44553)
0812 03:51:22.846279 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:22.846316 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":53,"threads_started":1}
I20260812 03:51:23.659860 2200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:23.659656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:51413)
0812 03:51:23.659780 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:23.659813 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260812 03:51:23.721429 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:23.721238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:58819)
0812 03:51:23.721356 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:23.721392 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260812 03:51:23.847074 2202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:23.846888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:45917)
0812 03:51:23.847005 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:23.847039 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260812 03:51:24.660594 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:24.660300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:35363)
0812 03:51:24.660513 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:24.660548 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":59,"threads_started":1}
I20260812 03:51:24.722034 2205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:24.721832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:52739)
0812 03:51:24.721964 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:24.721996 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260812 03:51:24.777340 2206 raft_consensus.cc:482] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:24.777436 2206 raft_consensus.cc:504] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:51:24.777647 2206 leader_election.cc:288] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857), 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
W20260812 03:51:24.778051 1549 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
W20260812 03:51:24.778131 1544 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
I20260812 03:51:24.778174 1544 leader_election.cc:302] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ede9777b894344ba4330905bad13ce; no voters: 58e444ce901f4511b9558675d03aed8a, 8daa756e125c4a758a2d0dc573c8317e
I20260812 03:51:24.778261 2206 raft_consensus.cc:2726] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:51:24.847714 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:24.847502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:50249)
0812 03:51:24.847637 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:24.847676 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260812 03:51:25.661319 2208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:25.661059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:45557)
0812 03:51:25.661241 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:25.661275 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":95,"threads_started":1}
I20260812 03:51:25.722779 2211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:25.722438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:43993)
0812 03:51:25.722643 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:25.722689 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":66,"threads_started":1}
I20260812 03:51:25.848342 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:25.848144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:60395)
0812 03:51:25.848273 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:25.848306 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260812 03:51:26.370954 2216 raft_consensus.cc:482] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:26.371042 2216 raft_consensus.cc:504] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58e444ce901f4511b9558675d03aed8a" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 34857 } } peers { permanent_uuid: "8daa756e125c4a758a2d0dc573c8317e" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 40681 } } peers { permanent_uuid: "37ede9777b894344ba4330905bad13ce" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45463 } }
I20260812 03:51:26.371243 2216 leader_election.cc:288] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857), 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681)
W20260812 03:51:26.371613 1544 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8daa756e125c4a758a2d0dc573c8317e (127.7.132.131:40681): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
W20260812 03:51:26.371680 1549 leader_election.cc:341] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58e444ce901f4511b9558675d03aed8a (127.7.132.130:34857): Not found: Tablet not found: 70d01a7184dd46a6bb2e3e3a267b13d4
I20260812 03:51:26.371769 1549 leader_election.cc:302] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 37ede9777b894344ba4330905bad13ce; no voters: 58e444ce901f4511b9558675d03aed8a, 8daa756e125c4a758a2d0dc573c8317e
I20260812 03:51:26.371838 2216 raft_consensus.cc:2726] T 70d01a7184dd46a6bb2e3e3a267b13d4 P 37ede9777b894344ba4330905bad13ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:51:26.661985 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:26.661776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:42669)
0812 03:51:26.661904 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:26.661939 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260812 03:51:26.723345 2219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:26.723157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:40943)
0812 03:51:26.723278 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:26.723310 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260812 03:51:26.848948 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:26.848736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:56877)
0812 03:51:26.848869 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:26.848904 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260812 03:51:27.662603 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:27.662417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:46535)
0812 03:51:27.662528 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:27.662560 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260812 03:51:27.723955 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:27.723727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:38179)
0812 03:51:27.723866 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:27.723900 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260812 03:51:27.849535 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:27.849349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:52275)
0812 03:51:27.849468 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:27.849497 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
I20260812 03:51:28.663196 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:28.662996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:35441)
0812 03:51:28.663121 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:28.663150 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":58,"threads_started":1}
I20260812 03:51:28.724623 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:28.724397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:44539)
0812 03:51:28.724539 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:28.724569 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260812 03:51:28.850246 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:28.849990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:35287)
0812 03:51:28.850130 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:28.850173 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":65,"threads_started":1}
I20260812 03:51:29.663890 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:29.663670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:46413)
0812 03:51:29.663808 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:29.663844 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260812 03:51:29.725322 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:29.725026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:41685)
0812 03:51:29.725243 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:29.725280 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":81,"threads_started":1}
I20260812 03:51:29.850994 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:29.850785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:49271)
0812 03:51:29.850916 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:29.850951 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260812 03:51:30.664801 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:30.664550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.129:49125)
0812 03:51:30.664702 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:30.664758 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260812 03:51:30.726207 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:30.725982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.130:48173)
0812 03:51:30.726119 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:30.726167 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260812 03:51:30.851748 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:30.851538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:43005 (local address 127.7.132.131:57377)
0812 03:51:30.851666 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:30.851708 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.190:43005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
I20260812 03:51:30.854557 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 1465
I20260812 03:51:30.866482 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 1729
I20260812 03:51:30.875993 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 1938
2026-08-12T03:51:30Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59439 ms)
[----------] 30 tests from SecurityITest (234363 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:31 dist-test-slave-39l8 krb5kdc[2247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:31 dist-test-slave-39l8 krb5kdc[2247](info): set up 2 sockets
Aug 12 03:51:31 dist-test-slave-39l8 krb5kdc[2247](info): commencing operation
krb5kdc: starting...
W20260812 03:51:32.791719 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:32 dist-test-slave-39l8 krb5kdc[2247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:51:32.906469 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42799
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:32.986294 2269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:32.986495 2269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:32.986516 2269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:32.987720 2269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:32.987768 2269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:32.987783 2269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:32.987826 2269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:32.987844 2269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:32.989368 2269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:32.989624 2269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:32.989850 2269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:51:32.994935 2269 server_base.cc:1034] running on GCE node
W20260812 03:51:32.994894 2282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:32.994911 2279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:32.994910 2278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:32.995330 2269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:32.995642 2269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:32.996794 2269 hybrid_clock.cc:648] HybridClock initialized: now 1786506692996788 us; error 37 us; skew 500 ppm
Aug 12 03:51:32 dist-test-slave-39l8 krb5kdc[2247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506692, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:32.999173 2269 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:51:32.999661 2269 webserver.cc:460] Webserver started at http://127.7.132.190:33023/ using document root <none> and password file <none>
I20260812 03:51:32.999894 2269 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:32.999969 2269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:33.000121 2269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:33.001457 2269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "ef8a73bac8944372aed39d50d5578cf1"
format_stamp: "Formatted at 2026-08-12 03:51:32 on dist-test-slave-39l8"
server_key: "283977ea31a9d027c2384d2bb847163f"
server_key_iv: "6a46090391fa29704d1dda7169c7cb64"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.001819 2269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "ef8a73bac8944372aed39d50d5578cf1"
format_stamp: "Formatted at 2026-08-12 03:51:32 on dist-test-slave-39l8"
server_key: "283977ea31a9d027c2384d2bb847163f"
server_key_iv: "6a46090391fa29704d1dda7169c7cb64"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.003193 2269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:51:33.003875 2305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.004037 2269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:33.004133 2269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "ef8a73bac8944372aed39d50d5578cf1"
format_stamp: "Formatted at 2026-08-12 03:51:32 on dist-test-slave-39l8"
server_key: "283977ea31a9d027c2384d2bb847163f"
server_key_iv: "6a46090391fa29704d1dda7169c7cb64"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.004211 2269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:33.019095 2269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:33.020992 2269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:33.021147 2269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:33.025239 2269 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42799
I20260812 03:51:33.025269 2389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42799 every 8 connection(s)
I20260812 03:51:33.025614 2269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:33.026508 2390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:33.028788 2390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1: Bootstrap starting.
I20260812 03:51:33.029306 2390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:33.029551 2390 log.cc:824] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:33.030213 2390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1: No bootstrap required, opened a new log
I20260812 03:51:33.030917 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 2269
I20260812 03:51:33.031032 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:33.031240 7698 external_mini_cluster.cc:1428] Setting key 02135dc01b83fa0de8126701926d3c15
I20260812 03:51:33.032264 2390 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8a73bac8944372aed39d50d5578cf1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42799 } }
I20260812 03:51:33.032454 2390 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:33.032500 2390 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef8a73bac8944372aed39d50d5578cf1, State: Initialized, Role: FOLLOWER
I20260812 03:51:33.032639 2390 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8a73bac8944372aed39d50d5578cf1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42799 } }
I20260812 03:51:33.032745 2390 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:33.032781 2390 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:33.032836 2390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:33.033844 2390 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8a73bac8944372aed39d50d5578cf1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42799 } }
I20260812 03:51:33.033964 2390 leader_election.cc:302] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef8a73bac8944372aed39d50d5578cf1; no voters:
I20260812 03:51:33.034143 2390 leader_election.cc:288] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:33.034189 2397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:33.034375 2397 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [term 1 LEADER]: Becoming Leader. State: Replica: ef8a73bac8944372aed39d50d5578cf1, State: Running, Role: LEADER
I20260812 03:51:33.034417 2390 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:33.034502 2397 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8a73bac8944372aed39d50d5578cf1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42799 } }
I20260812 03:51:33.034904 2397 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef8a73bac8944372aed39d50d5578cf1. Latest consensus state: current_term: 1 leader_uuid: "ef8a73bac8944372aed39d50d5578cf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8a73bac8944372aed39d50d5578cf1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42799 } } }
I20260812 03:51:33.035120 2397 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:33.035063 2399 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef8a73bac8944372aed39d50d5578cf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef8a73bac8944372aed39d50d5578cf1" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42799 } } }
I20260812 03:51:33.035347 2399 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1 [sys.catalog]: This master's current role is: LEADER
W20260812 03:51:33.036399 2421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:51:33.036458 2421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:51:33.036500 2410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:33.036631 2410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:51:33 dist-test-slave-39l8 krb5kdc[2247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:33.039099 2410 catalog_manager.cc:1347] Generated new cluster ID: 4fdf7355bf1f4bf2a1869dcbdecbdadf
I20260812 03:51:33.039147 2410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:33.042865 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.031820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45770 (local address 127.7.132.190:42799)
0812 03:51:33.031992 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.032000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.032034 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:51:33.032221 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.032226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.032272 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.032376 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:33.033255 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.034364 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.035504 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.036022 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.036115 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.038966 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:33.038970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:33.038971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:33.038983 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:33.041031 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.041395 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.041398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.041399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.041525 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.041716 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.041717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.041718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.042016 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:33.042189 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.042258 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.042493 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260812 03:51:33.067629 2410 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:33.068197 2410 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:33.074679 2410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1: Generated new TSK 0
I20260812 03:51:33.074892 2410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:51:33.107370 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42799
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:33.188704 2427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:33.188886 2427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:33.188907 2427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:33.189975 2427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:33.190014 2427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:33.190070 2427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:51:33.191540 2427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:33.191797 2427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:33.192042 2427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:33.196617 2435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:33.196655 2441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:33.196667 2439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:33.196704 2427 server_base.cc:1034] running on GCE node
I20260812 03:51:33.196939 2427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:33.197167 2427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:33.198333 2427 hybrid_clock.cc:648] HybridClock initialized: now 1786506693198318 us; error 49 us; skew 500 ppm
Aug 12 03:51:33 dist-test-slave-39l8 krb5kdc[2247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:33.200834 2427 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:51:33.201254 2427 webserver.cc:460] Webserver started at http://127.7.132.129:44491/ using document root <none> and password file <none>
I20260812 03:51:33.201474 2427 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:33.201535 2427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:33.201673 2427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:33.202638 2427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "158775180a1043ee9c1e1979121b5965"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "1b720c4c2289ebe77fac6f9c768e31c1"
server_key_iv: "aa21ea2c228d97715534c62c8ccfcf37"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.203035 2427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "158775180a1043ee9c1e1979121b5965"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "1b720c4c2289ebe77fac6f9c768e31c1"
server_key_iv: "aa21ea2c228d97715534c62c8ccfcf37"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.204236 2427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:33.204914 2462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.205089 2427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:33.205188 2427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "158775180a1043ee9c1e1979121b5965"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "1b720c4c2289ebe77fac6f9c768e31c1"
server_key_iv: "aa21ea2c228d97715534c62c8ccfcf37"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.205277 2427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:33.217499 2427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:33.219718 2427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:33.219879 2427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:33.220150 2427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:33.220463 2427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:33.220517 2427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.220559 2427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:33.220587 2427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.226114 2427 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:42639
I20260812 03:51:33.226166 2655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:42639 every 8 connection(s)
I20260812 03:51:33.226461 2427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:51:33 dist-test-slave-39l8 krb5kdc[2247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:33.232692 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 2427
I20260812 03:51:33.232812 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:51:33.233054 7698 external_mini_cluster.cc:1428] Setting key 3158266608a3c1cd558645b65ca41beb
I20260812 03:51:33.235473 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.227169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:51927 (local address 127.7.132.190:42799)
0812 03:51:33.227239 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.227242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.227432 (+ 190us) server_negotiation.cc:407] Connection header received
0812 03:51:33.227870 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.227871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.227884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.227933 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:33.228725 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.229177 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.230098 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.230299 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.230348 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.232381 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:33.232382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:33.232382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:33.232390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:33.233920 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.234405 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.234407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.234407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.234502 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.234834 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.234835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.234835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.234936 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:33.234989 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.235333 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.235397 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:51:33.235733 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.227139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42799 (local address 127.7.132.129:51927)
0812 03:51:33.227313 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.227347 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:33.227789 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:33.227978 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:33.227981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:33.228482 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:33.228653 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.228656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.229255 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.229256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.230022 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.230024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.230370 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.230371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.230465 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.231037 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:33.231044 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:33.232296 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:33.233997 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:33.234004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:33.234006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:33.234300 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:33.234571 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:33.234572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:33.234572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:33.234731 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:33.235001 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:33.235008 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:33.235164 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 03:51:33.235277 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:51:33.235438 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260812 03:51:33.236246 2656 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42799
I20260812 03:51:33.236334 2656 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:33.236507 2656 heartbeater.cc:511] Master 127.7.132.190:42799 requested a full tablet report, sending...
I20260812 03:51:33.237010 2334 ts_manager.cc:194] Registered new tserver with Master: 158775180a1043ee9c1e1979121b5965 (127.7.132.129:42639)
I20260812 03:51:33.237491 2334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:51927
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:51:33.257308 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42799
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:33.334213 2669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:33.334398 2669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:33.334456 2669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:33.335775 2669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:33.335822 2669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:33.335850 2669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:51:33.337216 2669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:33.337385 2669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:33.337545 2669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:33.342298 2681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:33.342300 2677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:33.342329 2678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:33.342584 2669 server_base.cc:1034] running on GCE node
I20260812 03:51:33.342912 2669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:33.343168 2669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:33.344326 2669 hybrid_clock.cc:648] HybridClock initialized: now 1786506693344297 us; error 42 us; skew 500 ppm
Aug 12 03:51:33 dist-test-slave-39l8 krb5kdc[2247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:33.346779 2669 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:51:33.347178 2669 webserver.cc:460] Webserver started at http://127.7.132.130:43805/ using document root <none> and password file <none>
I20260812 03:51:33.347419 2669 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:33.347486 2669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:33.347615 2669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:33.348542 2669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "ad3a07934d3e493bb7ff4c2143c66a42"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "bbd644820b44fec3960f9130758a8321"
server_key_iv: "805c4b8c8a368709489f77c2b7f15944"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.348874 2669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "ad3a07934d3e493bb7ff4c2143c66a42"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "bbd644820b44fec3960f9130758a8321"
server_key_iv: "805c4b8c8a368709489f77c2b7f15944"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.350109 2669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:51:33.350804 2691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.350994 2669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.351087 2669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "ad3a07934d3e493bb7ff4c2143c66a42"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "bbd644820b44fec3960f9130758a8321"
server_key_iv: "805c4b8c8a368709489f77c2b7f15944"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.351171 2669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:33.380508 2669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:33.382215 2669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:33.382369 2669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:33.382579 2669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:33.382882 2669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:33.382936 2669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.382967 2669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:33.382992 2669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.388370 2669 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:35515
I20260812 03:51:33.388419 2867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:35515 every 8 connection(s)
I20260812 03:51:33.388720 2669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:51:33.392014 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 2669
I20260812 03:51:33.392124 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:51:33.392376 7698 external_mini_cluster.cc:1428] Setting key 91fc6ea8216ed4e9bc25bb1a5fa0a90b
Aug 12 03:51:33 dist-test-slave-39l8 krb5kdc[2247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:33.398556 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.389405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42799 (local address 127.7.132.130:41123)
0812 03:51:33.389589 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.389624 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:33.390078 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:33.390313 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:33.390317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:33.390942 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:33.391122 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.391127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.391760 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.391760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.392802 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.392804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.393229 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.393230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.393356 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.393907 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:33.393915 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:33.395521 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:33.397111 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:33.397119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:33.397122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:33.397450 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:33.397650 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:33.397651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:33.397652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:33.397794 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:33.398046 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:33.398052 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:33.398215 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 03:51:33.398310 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:51:33.398419 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260812 03:51:33.398576 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.389417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:41123 (local address 127.7.132.190:42799)
0812 03:51:33.389471 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.389474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.389712 (+ 238us) server_negotiation.cc:407] Connection header received
0812 03:51:33.390168 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.390169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.390180 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.390233 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:33.391198 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.391683 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.392962 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.393147 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.393221 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.395636 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:33.395637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:33.395637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:33.395643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:33.397013 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.397528 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.397529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.397529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.397581 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.397887 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.397887 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.397887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.397982 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:33.398027 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.398361 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.398449 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:51:33.399053 2868 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42799
I20260812 03:51:33.399148 2868 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:33.399310 2868 heartbeater.cc:511] Master 127.7.132.190:42799 requested a full tablet report, sending...
I20260812 03:51:33.399719 2334 ts_manager.cc:194] Registered new tserver with Master: ad3a07934d3e493bb7ff4c2143c66a42 (127.7.132.130:35515)
I20260812 03:51:33.400116 2334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:41123
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:51:33.417325 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42799
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:33.498158 2875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:33.498345 2875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:33.498368 2875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:33.499476 2875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:33.499536 2875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:33.499562 2875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:51:33.500984 2875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:33.501171 2875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:33.501405 2875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:33.506055 2882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:33.506084 2881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:33.506093 2887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:33.506227 2875 server_base.cc:1034] running on GCE node
I20260812 03:51:33.506428 2875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:33.506628 2875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:33.507777 2875 hybrid_clock.cc:648] HybridClock initialized: now 1786506693507776 us; error 36 us; skew 500 ppm
Aug 12 03:51:33 dist-test-slave-39l8 krb5kdc[2247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:33.510308 2875 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:51:33.510807 2875 webserver.cc:460] Webserver started at http://127.7.132.131:33837/ using document root <none> and password file <none>
I20260812 03:51:33.510993 2875 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:33.511029 2875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:33.511124 2875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:33.512079 2875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "cdb94a8c88954974986f997966101bc6"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "7590056ce901dbb443839c3b6693ba49"
server_key_iv: "e3edd6998da7bdf5f2411d9654e44dee"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.512358 2875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "cdb94a8c88954974986f997966101bc6"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "7590056ce901dbb443839c3b6693ba49"
server_key_iv: "e3edd6998da7bdf5f2411d9654e44dee"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.513531 2875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:51:33.514246 2897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.514390 2875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:33.514474 2875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "cdb94a8c88954974986f997966101bc6"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "7590056ce901dbb443839c3b6693ba49"
server_key_iv: "e3edd6998da7bdf5f2411d9654e44dee"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.514525 2875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:33.534572 2875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:33.536842 2875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:33.537005 2875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:33.537230 2875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:33.537536 2875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:33.537568 2875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.537608 2875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:33.537633 2875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.543363 2875 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:41375
I20260812 03:51:33.543435 3074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:41375 every 8 connection(s)
I20260812 03:51:33.543745 2875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:51:33 dist-test-slave-39l8 krb5kdc[2247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506693, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:33.551980 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 2875
I20260812 03:51:33.552102 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:51:33.552309 7698 external_mini_cluster.cc:1428] Setting key 5fba2f46c32bf19e69a9b6114cb99063
I20260812 03:51:33.552321 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.544589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:56855 (local address 127.7.132.190:42799)
0812 03:51:33.544634 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.544636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.544824 (+ 188us) server_negotiation.cc:407] Connection header received
0812 03:51:33.545364 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.545365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.545378 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.545425 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:33.546317 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.546830 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.547772 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.547950 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.548020 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.549906 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:33.549907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:33.549907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:33.549914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:33.551155 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.551480 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.551481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.551481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.551526 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.551717 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.551718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.551718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.551806 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:33.551853 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.552195 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.552262 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:51:33.552561 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.544547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42799 (local address 127.7.132.131:56855)
0812 03:51:33.544699 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.544741 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:33.545255 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:33.545488 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:33.545492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:33.546028 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:33.546209 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.546214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.546917 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.546918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.547669 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.547671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.548027 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.548028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.548149 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.548555 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:33.548563 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:33.549821 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:33.551220 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:33.551226 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:33.551228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:33.551419 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:33.551580 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:33.551580 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:33.551581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:33.551667 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:33.551868 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:33.551875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:33.552051 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 03:51:33.552172 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 03:51:33.552334 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260812 03:51:33.553009 3075 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42799
I20260812 03:51:33.553100 3075 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:33.553267 3075 heartbeater.cc:511] Master 127.7.132.190:42799 requested a full tablet report, sending...
I20260812 03:51:33.553663 2334 ts_manager.cc:194] Registered new tserver with Master: cdb94a8c88954974986f997966101bc6 (127.7.132.131:41375)
I20260812 03:51:33.553999 2334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:56855
I20260812 03:51:33.564159 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:51:33.570881 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.565469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.7.132.190:42799)
0812 03:51:33.565540 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.565542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.565571 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:51:33.565634 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.565634 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.565642 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.565687 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:33.566219 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.566659 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.567363 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.567533 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.567581 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.568623 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:33.568624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:33.568624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:33.568630 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:33.569942 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.570236 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.570237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.570237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.570284 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:33.570430 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:33.570430 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:33.570430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:33.570546 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:33.570615 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.570698 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.570824 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:51:33.572314 2334 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:51:33.573817 2334 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:51:33.580013 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.576174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:41375 (local address 127.0.0.1:59346)
0812 03:51:33.576395 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.576407 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:33.576455 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:33.576983 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:33.576984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:33.576986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:33.577032 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.577033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.577876 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.577877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.578881 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.578884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.579723 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.579723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.579804 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.579819 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 03:51:33.579892 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:51:33.579940 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":32,"thread_start_us":56,"threads_started":1}
I20260812 03:51:33.580049 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.576352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.7.132.131:41375)
0812 03:51:33.576837 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.576841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.576850 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:51:33.576883 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.576885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.576898 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.576977 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:33.577106 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.577760 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.578960 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.579602 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.579715 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.579784 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.579923 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.579967 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":77,"threads_started":1}
I20260812 03:51:33.580188 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.576069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.7.132.130:35515)
0812 03:51:33.576266 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.576271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.576282 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:51:33.576322 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.576324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.576341 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.576426 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:33.576641 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.577708 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.579066 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.579719 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.579779 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.579825 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.580069 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.580114 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":77,"threads_started":1}
I20260812 03:51:33.580336 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.575948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:35515 (local address 127.0.0.1:43974)
0812 03:51:33.576072 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.576114 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:33.576177 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:33.576517 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:33.576520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:33.576521 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:33.576565 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.576566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.577820 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.577821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.578988 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.578991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.579860 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.579861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.579931 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.579935 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:51:33.579985 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:51:33.580026 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":53,"thread_start_us":74,"threads_started":1}
I20260812 03:51:33.581022 2767 tablet_service.cc:1467] Processing CreateTablet for tablet fca559b7f7834b8690648054a2a9b226 (DEFAULT_TABLE table=test-table [id=d82c9fd861914cdab23c0acc58cc88ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:33.581000 2971 tablet_service.cc:1467] Processing CreateTablet for tablet fca559b7f7834b8690648054a2a9b226 (DEFAULT_TABLE table=test-table [id=d82c9fd861914cdab23c0acc58cc88ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:33.581347 2971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca559b7f7834b8690648054a2a9b226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:33.581347 2767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca559b7f7834b8690648054a2a9b226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:33.583003 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.575993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42639 (local address 127.0.0.1:59444)
0812 03:51:33.576355 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.576367 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:33.576410 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:33.577037 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:33.577038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:33.577041 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:33.577082 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.577084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.578327 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.578328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.579441 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.579444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.582774 (+ 3330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.582775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.582834 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.582837 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:33.582906 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:51:33.582944 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":48,"threads_started":1}
I20260812 03:51:33.583046 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.576198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.7.132.129:42639)
0812 03:51:33.576539 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.576543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.576552 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:51:33.576614 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.576616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.576632 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.576709 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:33.577160 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.578221 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.579522 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.580634 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.580707 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.580764 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.582923 (+ 2159us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.582965 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":110,"threads_started":1}
I20260812 03:51:33.583978 2541 tablet_service.cc:1467] Processing CreateTablet for tablet fca559b7f7834b8690648054a2a9b226 (DEFAULT_TABLE table=test-table [id=d82c9fd861914cdab23c0acc58cc88ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:33.584295 2541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca559b7f7834b8690648054a2a9b226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:33.584556 3097 tablet_bootstrap.cc:492] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Bootstrap starting.
I20260812 03:51:33.584765 3096 tablet_bootstrap.cc:492] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: Bootstrap starting.
I20260812 03:51:33.585449 3096 tablet_bootstrap.cc:654] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:33.586087 3096 log.cc:824] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:33.586871 3096 tablet_bootstrap.cc:492] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: No bootstrap required, opened a new log
I20260812 03:51:33.586953 3096 ts_tablet_manager.cc:1397] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:33.586962 3097 tablet_bootstrap.cc:654] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:33.587626 3097 log.cc:824] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:33.587991 3099 tablet_bootstrap.cc:492] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965: Bootstrap starting.
I20260812 03:51:33.588379 3097 tablet_bootstrap.cc:492] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: No bootstrap required, opened a new log
I20260812 03:51:33.588449 3097 ts_tablet_manager.cc:1397] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:51:33.588624 3099 tablet_bootstrap.cc:654] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:33.588732 3096 raft_consensus.cc:348] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.588907 3096 raft_consensus.cc:374] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:33.588951 3096 raft_consensus.cc:729] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdb94a8c88954974986f997966101bc6, State: Initialized, Role: FOLLOWER
I20260812 03:51:33.589068 3096 consensus_queue.cc:260] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.589146 3099 log.cc:824] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:33.589340 3096 ts_tablet_manager.cc:1428] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:33.589504 3075 heartbeater.cc:503] Master 127.7.132.190:42799 was elected leader, sending a full tablet report...
I20260812 03:51:33.590075 3099 tablet_bootstrap.cc:492] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965: No bootstrap required, opened a new log
I20260812 03:51:33.590147 3099 ts_tablet_manager.cc:1397] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:33.590562 3097 raft_consensus.cc:348] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.590809 3097 raft_consensus.cc:374] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:33.590853 3097 raft_consensus.cc:729] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad3a07934d3e493bb7ff4c2143c66a42, State: Initialized, Role: FOLLOWER
I20260812 03:51:33.590966 3097 consensus_queue.cc:260] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.591240 3097 ts_tablet_manager.cc:1428] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:51:33.591351 2868 heartbeater.cc:503] Master 127.7.132.190:42799 was elected leader, sending a full tablet report...
I20260812 03:51:33.592217 3099 raft_consensus.cc:348] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.592361 3099 raft_consensus.cc:374] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:33.592382 3099 raft_consensus.cc:729] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 158775180a1043ee9c1e1979121b5965, State: Initialized, Role: FOLLOWER
I20260812 03:51:33.592490 3099 consensus_queue.cc:260] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.592706 3099 ts_tablet_manager.cc:1428] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:51:33.592754 2656 heartbeater.cc:503] Master 127.7.132.190:42799 was elected leader, sending a full tablet report...
W20260812 03:51:33.639283 2869 tablet.cc:2369] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:51:33.675709 3105 raft_consensus.cc:482] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:33.675839 3105 raft_consensus.cc:504] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.676297 3105 leader_election.cc:288] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cdb94a8c88954974986f997966101bc6 (127.7.132.131:41375), 158775180a1043ee9c1e1979121b5965 (127.7.132.129:42639)
I20260812 03:51:33.680195 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.676581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:54483 (local address 127.7.132.129:42639)
0812 03:51:33.676624 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.676625 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.676806 (+ 181us) server_negotiation.cc:407] Connection header received
0812 03:51:33.676841 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.676842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.676851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.676891 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:33.677043 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.677706 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.679236 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.679825 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.679899 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.679908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.680083 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.680123 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 03:51:33.680190 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.676486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:41375 (local address 127.7.132.130:35469)
0812 03:51:33.676544 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.676556 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:33.676599 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:33.676771 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:33.676772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:33.676774 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:33.676851 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.676852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.677795 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.677796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.679147 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.679157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.679910 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.679911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.679997 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.680000 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:33.680054 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:51:33.680092 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":89}
I20260812 03:51:33.680195 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.676583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:35469 (local address 127.7.132.131:41375)
0812 03:51:33.676634 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.676635 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.676645 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:51:33.676681 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.676682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.676690 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.676737 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:33.676946 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.677706 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.679239 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.679818 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.679899 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.679908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.680083 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.680123 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:51:33.680198 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.676502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42639 (local address 127.7.132.130:54483)
0812 03:51:33.676703 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.676710 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:33.676782 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:33.676914 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:33.676915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:33.676917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:33.676969 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.676970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.677795 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.677796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.679147 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:33.679150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.679910 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.679911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:33.679986 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.679990 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:51:33.680049 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:51:33.680091 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":84,"thread_start_us":114,"threads_started":1}
I20260812 03:51:33.680575 2575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca559b7f7834b8690648054a2a9b226" candidate_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158775180a1043ee9c1e1979121b5965" is_pre_election: true
I20260812 03:51:33.680713 2575 raft_consensus.cc:2454] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad3a07934d3e493bb7ff4c2143c66a42 in term 0.
I20260812 03:51:33.680657 3000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca559b7f7834b8690648054a2a9b226" candidate_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb94a8c88954974986f997966101bc6" is_pre_election: true
I20260812 03:51:33.680841 3000 raft_consensus.cc:2454] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad3a07934d3e493bb7ff4c2143c66a42 in term 0.
I20260812 03:51:33.680976 2701 leader_election.cc:302] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158775180a1043ee9c1e1979121b5965, ad3a07934d3e493bb7ff4c2143c66a42; no voters:
I20260812 03:51:33.681121 3105 raft_consensus.cc:2781] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:51:33.681198 3105 raft_consensus.cc:482] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:51:33.681216 3105 raft_consensus.cc:3037] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:33.681874 3105 raft_consensus.cc:504] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.682024 3105 leader_election.cc:288] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [CANDIDATE]: Term 1 election: Requested vote from peers cdb94a8c88954974986f997966101bc6 (127.7.132.131:41375), 158775180a1043ee9c1e1979121b5965 (127.7.132.129:42639)
I20260812 03:51:33.682188 3000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca559b7f7834b8690648054a2a9b226" candidate_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdb94a8c88954974986f997966101bc6"
I20260812 03:51:33.682201 2575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca559b7f7834b8690648054a2a9b226" candidate_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "158775180a1043ee9c1e1979121b5965"
I20260812 03:51:33.682277 3000 raft_consensus.cc:3037] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:33.682277 2575 raft_consensus.cc:3037] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:33.683257 2575 raft_consensus.cc:2454] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad3a07934d3e493bb7ff4c2143c66a42 in term 1.
I20260812 03:51:33.683257 3000 raft_consensus.cc:2454] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad3a07934d3e493bb7ff4c2143c66a42 in term 1.
I20260812 03:51:33.683451 2701 leader_election.cc:302] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158775180a1043ee9c1e1979121b5965, ad3a07934d3e493bb7ff4c2143c66a42; no voters:
I20260812 03:51:33.683569 3105 raft_consensus.cc:2781] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:33.683744 3105 raft_consensus.cc:686] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 1 LEADER]: Becoming Leader. State: Replica: ad3a07934d3e493bb7ff4c2143c66a42, State: Running, Role: LEADER
I20260812 03:51:33.683851 3105 consensus_queue.cc:237] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } }
I20260812 03:51:33.684741 2334 catalog_manager.cc:5495] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad3a07934d3e493bb7ff4c2143c66a42 (127.7.132.130). New cstate: current_term: 1 leader_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad3a07934d3e493bb7ff4c2143c66a42" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 35515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 } health_report { overall_health: UNKNOWN } } }
I20260812 03:51:33.693447 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.691476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.7.132.130:35515)
0812 03:51:33.691513 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.691515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.691523 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:33.691598 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.691599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.691614 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.691664 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:51:33.691785 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.692221 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.692877 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.693040 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.693086 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.693225 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:51:33.693306 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:51:33.693359 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.693388 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.693403 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:51:33.694932 3000 raft_consensus.cc:1261] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 1 FOLLOWER]: Refusing update from remote peer ad3a07934d3e493bb7ff4c2143c66a42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:33.694967 2575 raft_consensus.cc:1261] T fca559b7f7834b8690648054a2a9b226 P 158775180a1043ee9c1e1979121b5965 [term 1 FOLLOWER]: Refusing update from remote peer ad3a07934d3e493bb7ff4c2143c66a42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:33.695243 3108 consensus_queue.cc:1035] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdb94a8c88954974986f997966101bc6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:33.695353 3105 consensus_queue.cc:1035] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "158775180a1043ee9c1e1979121b5965" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:33.698282 3113 mvcc.cc:205] Tried to move back new op lower bound from 7317531417372184576 to 7317531417329836032. Current Snapshot: MvccSnapshot[applied={T|T < 7317531417372184576}]
I20260812 03:51:33.699692 2334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:33.699792 2334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:33.700503 2334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef8a73bac8944372aed39d50d5578cf1: Sending DeleteTablet for 3 replicas of tablet fca559b7f7834b8690648054a2a9b226
I20260812 03:51:33.700817 2767 tablet_service.cc:1514] Processing DeleteTablet for tablet fca559b7f7834b8690648054a2a9b226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:33 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:43974
I20260812 03:51:33.700884 2541 tablet_service.cc:1514] Processing DeleteTablet for tablet fca559b7f7834b8690648054a2a9b226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:33 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59444
I20260812 03:51:33.700887 2971 tablet_service.cc:1514] Processing DeleteTablet for tablet fca559b7f7834b8690648054a2a9b226 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:33 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:59346
I20260812 03:51:33.701105 3137 tablet_replica.cc:330] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: stopping tablet replica
I20260812 03:51:33.701201 3137 raft_consensus.cc:2229] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:51:33.701274 3134 tablet_replica.cc:330] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: stopping tablet replica
I20260812 03:51:33.701296 3137 raft_consensus.cc:2258] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:33.701359 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 2427
I20260812 03:51:33.701697 3137 ts_tablet_manager.cc:1905] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:33.707043 3137 ts_tablet_manager.cc:1918] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:33.707108 3137 log.cc:1196] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/fca559b7f7834b8690648054a2a9b226
I20260812 03:51:33.707366 3137 ts_tablet_manager.cc:1939] T fca559b7f7834b8690648054a2a9b226 P cdb94a8c88954974986f997966101bc6: Deleting consensus metadata
I20260812 03:51:33.713009 2310 catalog_manager.cc:4841] TS cdb94a8c88954974986f997966101bc6 (127.7.132.131:41375): tablet fca559b7f7834b8690648054a2a9b226 (table test-table [id=d82c9fd861914cdab23c0acc58cc88ee]) successfully deleted
I20260812 03:51:33.714916 3134 raft_consensus.cc:2229] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:51:33.715106 3134 raft_consensus.cc:2258] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:33.715488 3134 ts_tablet_manager.cc:1905] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:33.716794 3134 ts_tablet_manager.cc:1918] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:33.716874 3134 log.cc:1196] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/fca559b7f7834b8690648054a2a9b226
I20260812 03:51:33.717126 3134 ts_tablet_manager.cc:1939] T fca559b7f7834b8690648054a2a9b226 P ad3a07934d3e493bb7ff4c2143c66a42: Deleting consensus metadata
I20260812 03:51:33.717474 2310 catalog_manager.cc:4841] TS ad3a07934d3e493bb7ff4c2143c66a42 (127.7.132.130:35515): tablet fca559b7f7834b8690648054a2a9b226 (table test-table [id=d82c9fd861914cdab23c0acc58cc88ee]) successfully deleted
W20260812 03:51:33.721493 2311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:42639 (error 108)
I20260812 03:51:33.722010 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.721855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:42639 (local address 127.0.0.1:59456)
0812 03:51:33.721911 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:33.721948 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:42639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 03:51:33.722078 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 2669
W20260812 03:51:33.726178 2311 catalog_manager.cc:4584] TS 158775180a1043ee9c1e1979121b5965 (127.7.132.129:42639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fca559b7f7834b8690648054a2a9b226: Network error: Client connection negotiation failed: client connection to 127.7.132.129:42639: connect: Connection refused (error 111)
I20260812 03:51:33.730957 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 2875
I20260812 03:51:33.743655 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 2269
2026-08-12T03:51:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2129 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T03:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:33Z Disabled control of system clock
I20260812 03:51:33.776839 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46147
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:46147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:33.853001 3143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:33.853212 3143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:33.853251 3143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:33.854275 3143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:33.854312 3143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:33.854324 3143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:33.854336 3143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:33.854383 3143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:33.855620 3143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:46147
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:46147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:33.855875 3143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:33.856164 3143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:33.860893 3149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:33.860893 3150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:33.860888 3153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:33.861160 3143 server_base.cc:1034] running on GCE node
I20260812 03:51:33.861356 3143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:33.861563 3143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:33.862689 3143 hybrid_clock.cc:648] HybridClock initialized: now 1786506693862678 us; error 28 us; skew 500 ppm
I20260812 03:51:33.863824 3143 webserver.cc:460] Webserver started at http://127.7.132.190:45207/ using document root <none> and password file <none>
I20260812 03:51:33.864061 3143 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:33.864105 3143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:33.864255 3143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:33.865173 3143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "7c56898871396b7f8a4ca5c16635d086"
server_key_iv: "363a8ea5a6e3998765b9c4b96f19787f"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.865512 3143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "7c56898871396b7f8a4ca5c16635d086"
server_key_iv: "363a8ea5a6e3998765b9c4b96f19787f"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.866659 3143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:51:33.867344 3163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:33.867506 3143 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:33.867587 3143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68"
format_stamp: "Formatted at 2026-08-12 03:51:33 on dist-test-slave-39l8"
server_key: "7c56898871396b7f8a4ca5c16635d086"
server_key_iv: "363a8ea5a6e3998765b9c4b96f19787f"
server_key_version: "encryptionkey@0"
I20260812 03:51:33.867657 3143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:33.882588 3143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:33.882892 3143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:33.883030 3143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:33.886741 3143 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:46147
I20260812 03:51:33.886766 3262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:46147 every 8 connection(s)
I20260812 03:51:33.887068 3143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:33.887938 3263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:33.889999 3263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68: Bootstrap starting.
I20260812 03:51:33.890575 3263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:33.890944 3263 log.cc:824] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:33.891271 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 3143
I20260812 03:51:33.891383 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:33.891600 7698 external_mini_cluster.cc:1428] Setting key 567ca3a25b134155a0668feb4c1ffaac
I20260812 03:51:33.891688 3263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68: No bootstrap required, opened a new log
I20260812 03:51:33.894134 3263 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46147 } }
I20260812 03:51:33.894310 3263 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:33.894346 3263 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2d4b580c0d4eeea0b9a939aa56cb68, State: Initialized, Role: FOLLOWER
I20260812 03:51:33.894455 3263 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46147 } }
I20260812 03:51:33.894524 3263 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:33.894552 3263 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:33.894590 3263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:33.895222 3263 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46147 } }
I20260812 03:51:33.895308 3263 leader_election.cc:302] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a2d4b580c0d4eeea0b9a939aa56cb68; no voters:
I20260812 03:51:33.895476 3263 leader_election.cc:288] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:33.895504 3273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:33.895604 3273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [term 1 LEADER]: Becoming Leader. State: Replica: 1a2d4b580c0d4eeea0b9a939aa56cb68, State: Running, Role: LEADER
I20260812 03:51:33.895682 3273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46147 } }
I20260812 03:51:33.895730 3263 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:33.895963 3273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a2d4b580c0d4eeea0b9a939aa56cb68. Latest consensus state: current_term: 1 leader_uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46147 } } }
I20260812 03:51:33.895974 3274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2d4b580c0d4eeea0b9a939aa56cb68" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 46147 } } }
I20260812 03:51:33.896039 3273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:33.896063 3274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:33.896375 3277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:33.896824 3277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:33.897286 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:33.892207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.7.132.190:46147)
0812 03:51:33.892394 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:33.892400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:33.892421 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:51:33.892994 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:33.893001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:33.893053 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:33.893157 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:33.893378 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.894140 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.895150 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:33.895676 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:33.896076 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:33.896448 (+ 372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:33.896452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:33.896452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:33.896656 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:33.896697 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:33.896712 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:33.896772 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:33.896964 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 03:51:33.897044 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260812 03:51:33.898403 3277 catalog_manager.cc:1347] Generated new cluster ID: 426f09df68e940228006a3581cc5f6de
I20260812 03:51:33.898451 3277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:33.918921 3277 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:33.919613 3277 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:33.938308 3277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68: Generated new TSK 0
I20260812 03:51:33.938499 3277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:33.942472 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46147
--builtin_ntp_servers=127.7.132.148:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:34.027362 3301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:34.027660 3301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:34.027695 3301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:34.028705 3301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:34.028771 3301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:34.028810 3301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:51:34.030025 3301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:34.030282 3301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:34.030591 3301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:34.035455 3318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:34.035482 3314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:34.035614 3301 server_base.cc:1034] running on GCE node
W20260812 03:51:34.035672 3309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:34.035889 3301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:34.036082 3301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:34.037209 3301 hybrid_clock.cc:648] HybridClock initialized: now 1786506694037199 us; error 27 us; skew 500 ppm
I20260812 03:51:34.038450 3301 webserver.cc:460] Webserver started at http://127.7.132.129:42517/ using document root <none> and password file <none>
I20260812 03:51:34.038637 3301 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:34.038676 3301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:34.038820 3301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:34.039757 3301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "66d7a5e895214a27bc51ebd5662da709"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "df65d37810e7119f16d737fa3bf2923f"
server_key_iv: "ab124db8d1e69dfb0fa372f90a7f7121"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.040053 3301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "66d7a5e895214a27bc51ebd5662da709"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "df65d37810e7119f16d737fa3bf2923f"
server_key_iv: "ab124db8d1e69dfb0fa372f90a7f7121"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.041214 3301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:51:34.041791 3328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.041944 3301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:34.042022 3301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "66d7a5e895214a27bc51ebd5662da709"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "df65d37810e7119f16d737fa3bf2923f"
server_key_iv: "ab124db8d1e69dfb0fa372f90a7f7121"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.042073 3301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:34.053723 3301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:34.053913 3301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:34.053999 3301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:34.054136 3301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:34.054364 3301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:34.054391 3301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.054409 3301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:34.054420 3301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.059880 3301 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:37517
I20260812 03:51:34.059933 3536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:37517 every 8 connection(s)
I20260812 03:51:34.060226 3301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:51:34.065069 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.060878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:42701 (local address 127.7.132.190:46147)
0812 03:51:34.060934 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.060936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.061156 (+ 220us) server_negotiation.cc:407] Connection header received
0812 03:51:34.061619 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.061620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.061637 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.061685 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:34.062380 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.062964 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.063861 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.064045 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.064115 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.064800 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:34.064801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:34.064802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:34.064807 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:34.064821 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:34.064824 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:34.064880 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.064973 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.064997 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:51:34.065258 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.060858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46147 (local address 127.7.132.129:42701)
0812 03:51:34.061054 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.061082 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:34.061512 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:34.061748 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:34.061752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:34.061760 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:34.062283 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.062287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.063054 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.063055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.063775 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.063777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.064125 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.064126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.064247 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.064662 (+ 415us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:34.064663 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:34.064686 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:34.064704 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:34.064893 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:34.064898 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:34.064901 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:34.064975 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:51:34.065062 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260812 03:51:34.065675 3537 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46147
I20260812 03:51:34.065758 3537 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:34.065882 3537 heartbeater.cc:511] Master 127.7.132.190:46147 requested a full tablet report, sending...
I20260812 03:51:34.066336 3193 ts_manager.cc:194] Registered new tserver with Master: 66d7a5e895214a27bc51ebd5662da709 (127.7.132.129:37517)
I20260812 03:51:34.066942 3193 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.129:42701
I20260812 03:51:34.067948 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 3301
I20260812 03:51:34.068060 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:51:34.068269 7698 external_mini_cluster.cc:1428] Setting key f54ff9523acd3bb53cfd1dd011d8b815
I20260812 03:51:34.069104 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46147
--builtin_ntp_servers=127.7.132.148:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:34.148332 3542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:34.148514 3542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:34.148535 3542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:34.149479 3542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:34.149509 3542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:34.149531 3542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:51:34.150836 3542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:34.151026 3542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:34.151262 3542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:34.156002 3553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:34.156018 3550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:34.156107 3542 server_base.cc:1034] running on GCE node
W20260812 03:51:34.156021 3556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:34.156415 3542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:34.156648 3542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:34.157804 3542 hybrid_clock.cc:648] HybridClock initialized: now 1786506694157783 us; error 50 us; skew 500 ppm
I20260812 03:51:34.158962 3542 webserver.cc:460] Webserver started at http://127.7.132.130:45807/ using document root <none> and password file <none>
I20260812 03:51:34.159199 3542 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:34.159265 3542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:34.159407 3542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:34.160429 3542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "dbcc5663a25d1a23367bd8832b88ad64"
server_key_iv: "13b9eb702ac0cb4ae9df35727fd4a75b"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.160796 3542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "dbcc5663a25d1a23367bd8832b88ad64"
server_key_iv: "13b9eb702ac0cb4ae9df35727fd4a75b"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.162190 3542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:51:34.162870 3566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.163038 3542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:34.163137 3542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "dbcc5663a25d1a23367bd8832b88ad64"
server_key_iv: "13b9eb702ac0cb4ae9df35727fd4a75b"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.163228 3542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:34.176466 3542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:34.176709 3542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:34.176841 3542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:34.177031 3542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:34.177321 3542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:34.177372 3542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.177408 3542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:34.177435 3542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.182977 3542 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:44061
I20260812 03:51:34.183013 3763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:44061 every 8 connection(s)
I20260812 03:51:34.183321 3542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:51:34.184373 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 3542
I20260812 03:51:34.184468 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:51:34.184671 7698 external_mini_cluster.cc:1428] Setting key f1e67c49887730091c51f2a901a2874e
I20260812 03:51:34.185555 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46147
--builtin_ntp_servers=127.7.132.148:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:51:34.188658 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.184040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:59175 (local address 127.7.132.190:46147)
0812 03:51:34.184114 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.184116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.184362 (+ 246us) server_negotiation.cc:407] Connection header received
0812 03:51:34.184946 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.184948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.184959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.185030 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:34.185781 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.186224 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.187341 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.187512 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.187573 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.188384 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:34.188386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:34.188386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:34.188390 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:34.188406 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:34.188410 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:34.188464 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.188527 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.188551 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:51:34.188789 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.184029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46147 (local address 127.7.132.130:59175)
0812 03:51:34.184220 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.184258 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:34.184826 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:34.185096 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:34.185100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:34.185109 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:34.185694 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.185702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.186325 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.186326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.187259 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.187263 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.187591 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.187592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.187743 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.188260 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:34.188261 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:34.188286 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:34.188306 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:34.188473 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:34.188476 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:34.188478 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:51:34.188525 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:51:34.188604 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260812 03:51:34.189217 3764 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46147
I20260812 03:51:34.189321 3764 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:34.189488 3764 heartbeater.cc:511] Master 127.7.132.190:46147 requested a full tablet report, sending...
I20260812 03:51:34.189894 3193 ts_manager.cc:194] Registered new tserver with Master: 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 (127.7.132.130:44061)
I20260812 03:51:34.190429 3193 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.130:59175
W20260812 03:51:34.263805 3769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:34.263968 3769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:34.263986 3769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:34.264950 3769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:34.264977 3769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:34.264998 3769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:51:34.266268 3769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:46147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:34.266511 3769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:34.266813 3769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:34.271611 3780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:34.271579 3782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:34.271767 3769 server_base.cc:1034] running on GCE node
W20260812 03:51:34.271812 3778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:34.272056 3769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:34.272298 3769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:34.273456 3769 hybrid_clock.cc:648] HybridClock initialized: now 1786506694273443 us; error 42 us; skew 500 ppm
I20260812 03:51:34.274777 3769 webserver.cc:460] Webserver started at http://127.7.132.131:33925/ using document root <none> and password file <none>
I20260812 03:51:34.275017 3769 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:34.275082 3769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:34.275252 3769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:34.276232 3769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "cf25a8ecd16c47b995d657d5acfc10cb"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "40a3b2ed8173952486b08033cf7a93db"
server_key_iv: "7749b527891a82be7eb53f6d59b89f9c"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.276566 3769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "cf25a8ecd16c47b995d657d5acfc10cb"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "40a3b2ed8173952486b08033cf7a93db"
server_key_iv: "7749b527891a82be7eb53f6d59b89f9c"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.277755 3769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:51:34.278389 3790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.278546 3769 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:34.278646 3769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "cf25a8ecd16c47b995d657d5acfc10cb"
format_stamp: "Formatted at 2026-08-12 03:51:34 on dist-test-slave-39l8"
server_key: "40a3b2ed8173952486b08033cf7a93db"
server_key_iv: "7749b527891a82be7eb53f6d59b89f9c"
server_key_version: "encryptionkey@0"
I20260812 03:51:34.278754 3769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:34.305630 3769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:34.305977 3769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:34.306142 3769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:34.306402 3769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:34.306789 3769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:34.306844 3769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.306885 3769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:34.306919 3769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:34.312372 3769 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:43585
I20260812 03:51:34.312445 3988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:43585 every 8 connection(s)
I20260812 03:51:34.312769 3769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:51:34.317510 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.313488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:44681 (local address 127.7.132.190:46147)
0812 03:51:34.313550 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.313552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.313799 (+ 247us) server_negotiation.cc:407] Connection header received
0812 03:51:34.314256 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.314256 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.314277 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.314335 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:34.315051 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.315515 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.316397 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.316555 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.316599 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.317300 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:34.317301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:34.317302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:34.317305 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:34.317317 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:34.317321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:34.317360 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.317438 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.317457 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:51:34.317677 3991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.313487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:46147 (local address 127.7.132.131:44681)
0812 03:51:34.313662 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.313702 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:34.314172 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:34.314411 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:34.314414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:34.314422 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:34.314972 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.314977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.315589 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.315590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.316316 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.316318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.316620 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.316621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.316717 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.317178 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:34.317178 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:34.317205 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:34.317225 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:34.317371 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:34.317375 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:34.317377 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:51:34.317425 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 03:51:34.317495 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260812 03:51:34.318085 3989 heartbeater.cc:347] Connected to a master server at 127.7.132.190:46147
I20260812 03:51:34.318205 3989 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:34.318392 3989 heartbeater.cc:511] Master 127.7.132.190:46147 requested a full tablet report, sending...
I20260812 03:51:34.318768 3193 ts_manager.cc:194] Registered new tserver with Master: cf25a8ecd16c47b995d657d5acfc10cb (127.7.132.131:43585)
I20260812 03:51:34.319116 3193 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.131:44681
I20260812 03:51:34.321447 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 3769
I20260812 03:51:34.321520 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:51:34.321732 7698 external_mini_cluster.cc:1428] Setting key 6a8998c7ab59bf0eac9aaa19e550b9f1
I20260812 03:51:34.322496 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:51:34.326686 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.323864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.7.132.190:46147)
0812 03:51:34.323936 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.323937 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.324045 (+ 108us) server_negotiation.cc:407] Connection header received
0812 03:51:34.324119 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.324120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.324130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.324172 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:34.324343 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.324774 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.325510 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.325725 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.325809 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.326458 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:34.326459 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:34.326459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:34.326463 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:34.326475 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:34.326478 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:34.326528 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.326592 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.326610 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:51:34.328130 3193 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:51:34.329665 3193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:51:34.335480 4000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.331846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:43585 (local address 127.0.0.1:40790)
0812 03:51:34.331963 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.331991 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:34.332084 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:34.332299 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:34.332302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:34.332304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:34.332360 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.332361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.333493 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.333494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.334316 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.334318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.335192 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.335193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.335274 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.335288 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:51:34.335366 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:51:34.335415 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":80,"threads_started":1}
I20260812 03:51:34.335733 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.331939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.7.132.131:43585)
0812 03:51:34.332144 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.332148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.332157 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:51:34.332189 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.332190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.332203 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.332293 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:34.332550 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.333415 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.334410 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.335106 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.335176 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.335262 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.335587 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.335639 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260812 03:51:34.336347 4002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.332095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:37517 (local address 127.0.0.1:39782)
0812 03:51:34.332236 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.332245 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:34.332290 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:34.332861 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:34.332862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:34.332864 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:34.332912 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.332914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.334221 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.334222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.335301 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.335304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.336116 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.336117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.336174 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.336177 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:34.336231 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:51:34.336271 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":28,"thread_start_us":96,"threads_started":1}
I20260812 03:51:34.336386 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.332494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39782 (local address 127.7.132.129:37517)
0812 03:51:34.332699 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.332704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.332713 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:51:34.332751 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.332753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.332767 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.332848 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:34.332998 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.334113 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.335379 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.336048 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.336115 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.336196 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.336252 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.336286 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":99,"threads_started":1}
I20260812 03:51:34.336486 3880 tablet_service.cc:1467] Processing CreateTablet for tablet 64bd7ae7c2494015a2aef74ccc86ebb9 (DEFAULT_TABLE table=test-table [id=9850ed29dc264da38ee94a5c0d9a5b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:34.336935 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.332288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:44061 (local address 127.0.0.1:38404)
0812 03:51:34.332391 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.332401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:34.332440 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:34.332962 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:34.332963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:34.332965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:34.333006 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.333007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.334242 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.334243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.335178 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.335182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.336697 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.336698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.336758 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.336762 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:51:34.336831 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:51:34.336869 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
I20260812 03:51:34.336939 3880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bd7ae7c2494015a2aef74ccc86ebb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:34.336963 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.332575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.7.132.130:44061)
0812 03:51:34.332789 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.332792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.332800 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:34.332857 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.332858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.332873 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.332952 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:34.333080 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.334128 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.335276 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.336243 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.336314 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.336645 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.336848 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.336886 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":103,"threads_started":1}
I20260812 03:51:34.337054 3416 tablet_service.cc:1467] Processing CreateTablet for tablet 64bd7ae7c2494015a2aef74ccc86ebb9 (DEFAULT_TABLE table=test-table [id=9850ed29dc264da38ee94a5c0d9a5b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:34.337460 3416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bd7ae7c2494015a2aef74ccc86ebb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:34.337829 3648 tablet_service.cc:1467] Processing CreateTablet for tablet 64bd7ae7c2494015a2aef74ccc86ebb9 (DEFAULT_TABLE table=test-table [id=9850ed29dc264da38ee94a5c0d9a5b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:34.338138 3648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bd7ae7c2494015a2aef74ccc86ebb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:34.339525 4008 tablet_bootstrap.cc:492] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: Bootstrap starting.
I20260812 03:51:34.340286 4008 tablet_bootstrap.cc:654] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:34.340654 4008 log.cc:824] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:34.340682 4010 tablet_bootstrap.cc:492] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709: Bootstrap starting.
I20260812 03:51:34.341325 4012 tablet_bootstrap.cc:492] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Bootstrap starting.
I20260812 03:51:34.341431 4008 tablet_bootstrap.cc:492] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: No bootstrap required, opened a new log
I20260812 03:51:34.341503 4008 ts_tablet_manager.cc:1397] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:34.341614 4010 tablet_bootstrap.cc:654] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:34.341979 4012 tablet_bootstrap.cc:654] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:34.341979 4010 log.cc:824] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:34.342291 4012 log.cc:824] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:34.342818 4010 tablet_bootstrap.cc:492] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709: No bootstrap required, opened a new log
I20260812 03:51:34.342893 4010 ts_tablet_manager.cc:1397] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:34.343041 4012 tablet_bootstrap.cc:492] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: No bootstrap required, opened a new log
I20260812 03:51:34.343132 4012 ts_tablet_manager.cc:1397] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:34.344238 4008 raft_consensus.cc:348] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.344411 4008 raft_consensus.cc:374] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:34.344449 4008 raft_consensus.cc:729] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf25a8ecd16c47b995d657d5acfc10cb, State: Initialized, Role: FOLLOWER
I20260812 03:51:34.344591 4008 consensus_queue.cc:260] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.344861 4008 ts_tablet_manager.cc:1428] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 03:51:34.344975 3989 heartbeater.cc:503] Master 127.7.132.190:46147 was elected leader, sending a full tablet report...
I20260812 03:51:34.345502 4012 raft_consensus.cc:348] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.345496 4010 raft_consensus.cc:348] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.345645 4012 raft_consensus.cc:374] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:34.345644 4010 raft_consensus.cc:374] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:34.345691 4012 raft_consensus.cc:729] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a7e6b89c6f94bd7a6bb0e8985d2eeb9, State: Initialized, Role: FOLLOWER
I20260812 03:51:34.345692 4010 raft_consensus.cc:729] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d7a5e895214a27bc51ebd5662da709, State: Initialized, Role: FOLLOWER
I20260812 03:51:34.345803 4010 consensus_queue.cc:260] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.345808 4012 consensus_queue.cc:260] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.346052 4010 ts_tablet_manager.cc:1428] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:51:34.346060 4012 ts_tablet_manager.cc:1428] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:51:34.346167 3537 heartbeater.cc:503] Master 127.7.132.190:46147 was elected leader, sending a full tablet report...
I20260812 03:51:34.346207 3764 heartbeater.cc:503] Master 127.7.132.190:46147 was elected leader, sending a full tablet report...
W20260812 03:51:34.433940 3766 tablet.cc:2369] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:51:34.480554 4019 raft_consensus.cc:482] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:34.480685 4019 raft_consensus.cc:504] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.481125 4019 leader_election.cc:288] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf25a8ecd16c47b995d657d5acfc10cb (127.7.132.131:43585), 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 (127.7.132.130:44061)
I20260812 03:51:34.484040 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.481386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:34391 (local address 127.7.132.131:43585)
0812 03:51:34.481449 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.481451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.481492 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:51:34.481517 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.481518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.481525 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.481587 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:34.481788 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.482290 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.483293 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.483696 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.483780 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.483789 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.483935 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.483973 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:51:34.484040 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.481329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:43585 (local address 127.7.132.129:34391)
0812 03:51:34.481386 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.481398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:34.481443 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:34.481641 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:34.481643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:34.481645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:34.481712 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.481714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.482355 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.482356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.483205 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.483207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.483790 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.483791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.483863 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.483866 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:34.483929 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:51:34.483971 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:51:34.484378 3908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bd7ae7c2494015a2aef74ccc86ebb9" candidate_uuid: "66d7a5e895214a27bc51ebd5662da709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" is_pre_election: true
I20260812 03:51:34.484560 3908 raft_consensus.cc:2454] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66d7a5e895214a27bc51ebd5662da709 in term 0.
I20260812 03:51:34.484839 4026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.481437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:44061 (local address 127.7.132.129:57745)
0812 03:51:34.481600 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.481608 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:34.481653 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:34.482367 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:34.482368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:34.482369 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:34.482429 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.482430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.483088 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.483089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.483835 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:34.483837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.484526 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.484527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:34.484600 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.484602 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:51:34.484704 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 03:51:34.484761 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260812 03:51:34.484871 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.481589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:57745 (local address 127.7.132.130:44061)
0812 03:51:34.481641 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.481643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.481651 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:34.481707 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.481708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.481716 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.481769 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:34.482519 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.483014 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.483923 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.484420 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.484498 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.484506 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.484756 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.484799 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:51:34.484826 3333 leader_election.cc:302] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66d7a5e895214a27bc51ebd5662da709, cf25a8ecd16c47b995d657d5acfc10cb; no voters:
I20260812 03:51:34.485045 4019 raft_consensus.cc:2781] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:51:34.485114 4019 raft_consensus.cc:482] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:51:34.485141 4019 raft_consensus.cc:3037] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:34.485271 3684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bd7ae7c2494015a2aef74ccc86ebb9" candidate_uuid: "66d7a5e895214a27bc51ebd5662da709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" is_pre_election: true
I20260812 03:51:34.485416 3684 raft_consensus.cc:2454] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66d7a5e895214a27bc51ebd5662da709 in term 0.
I20260812 03:51:34.485932 4019 raft_consensus.cc:504] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.486060 4019 leader_election.cc:288] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [CANDIDATE]: Term 1 election: Requested vote from peers cf25a8ecd16c47b995d657d5acfc10cb (127.7.132.131:43585), 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 (127.7.132.130:44061)
I20260812 03:51:34.486212 3908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bd7ae7c2494015a2aef74ccc86ebb9" candidate_uuid: "66d7a5e895214a27bc51ebd5662da709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf25a8ecd16c47b995d657d5acfc10cb"
I20260812 03:51:34.486245 3684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bd7ae7c2494015a2aef74ccc86ebb9" candidate_uuid: "66d7a5e895214a27bc51ebd5662da709" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9"
I20260812 03:51:34.486291 3908 raft_consensus.cc:3037] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:34.486300 3684 raft_consensus.cc:3037] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:34.487188 3908 raft_consensus.cc:2454] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66d7a5e895214a27bc51ebd5662da709 in term 1.
I20260812 03:51:34.487188 3684 raft_consensus.cc:2454] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66d7a5e895214a27bc51ebd5662da709 in term 1.
I20260812 03:51:34.487365 3333 leader_election.cc:302] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66d7a5e895214a27bc51ebd5662da709, cf25a8ecd16c47b995d657d5acfc10cb; no voters:
I20260812 03:51:34.487448 4019 raft_consensus.cc:2781] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:34.487600 4019 raft_consensus.cc:686] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [term 1 LEADER]: Becoming Leader. State: Replica: 66d7a5e895214a27bc51ebd5662da709, State: Running, Role: LEADER
I20260812 03:51:34.487696 4019 consensus_queue.cc:237] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } }
I20260812 03:51:34.488525 3193 catalog_manager.cc:5495] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66d7a5e895214a27bc51ebd5662da709 (127.7.132.129). New cstate: current_term: 1 leader_uuid: "66d7a5e895214a27bc51ebd5662da709" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66d7a5e895214a27bc51ebd5662da709" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 37517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 } health_report { overall_health: UNKNOWN } } }
I20260812 03:51:34.515308 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.513111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.7.132.129:37517)
0812 03:51:34.513170 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:34.513172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:34.513183 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:51:34.513220 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:34.513220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:34.513251 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:34.513328 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:51:34.513462 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.513893 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.514565 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:34.514764 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:34.514837 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:34.514982 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:51:34.515075 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:51:34.515164 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:34.515187 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 03:51:34.515205 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:51:34.516891 3908 raft_consensus.cc:1261] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 1 FOLLOWER]: Refusing update from remote peer 66d7a5e895214a27bc51ebd5662da709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:34.516924 3684 raft_consensus.cc:1261] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 1 FOLLOWER]: Refusing update from remote peer 66d7a5e895214a27bc51ebd5662da709: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:34.517225 4027 consensus_queue.cc:1035] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf25a8ecd16c47b995d657d5acfc10cb" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:34.517349 4019 consensus_queue.cc:1035] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 66d7a5e895214a27bc51ebd5662da709 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a7e6b89c6f94bd7a6bb0e8985d2eeb9" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 44061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:34.521871 3193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:34.521966 3193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:34.522708 3193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a2d4b580c0d4eeea0b9a939aa56cb68: Sending DeleteTablet for 3 replicas of tablet 64bd7ae7c2494015a2aef74ccc86ebb9
I20260812 03:51:34.523079 3648 tablet_service.cc:1514] Processing DeleteTablet for tablet 64bd7ae7c2494015a2aef74ccc86ebb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:34 UTC) from {username='slave'} at 127.0.0.1:38404
I20260812 03:51:34.523079 3880 tablet_service.cc:1514] Processing DeleteTablet for tablet 64bd7ae7c2494015a2aef74ccc86ebb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:34 UTC) from {username='slave'} at 127.0.0.1:40790
I20260812 03:51:34.523321 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 3301
I20260812 03:51:34.523355 4051 tablet_replica.cc:330] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: stopping tablet replica
I20260812 03:51:34.523370 3416 tablet_service.cc:1514] Processing DeleteTablet for tablet 64bd7ae7c2494015a2aef74ccc86ebb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:34 UTC) from {username='slave'} at 127.0.0.1:39782
I20260812 03:51:34.523466 4051 raft_consensus.cc:2229] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:51:34.523550 4051 raft_consensus.cc:2258] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:34.523674 4052 tablet_replica.cc:330] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: stopping tablet replica
I20260812 03:51:34.523741 4052 raft_consensus.cc:2229] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:51:34.526074 4052 raft_consensus.cc:2258] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:34.527503 4051 ts_tablet_manager.cc:1905] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:34.527498 4052 ts_tablet_manager.cc:1905] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:34.529081 4051 ts_tablet_manager.cc:1918] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:34.529150 4051 log.cc:1196] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/64bd7ae7c2494015a2aef74ccc86ebb9
I20260812 03:51:34.529340 4052 ts_tablet_manager.cc:1918] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:34.529386 4052 log.cc:1196] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/64bd7ae7c2494015a2aef74ccc86ebb9
I20260812 03:51:34.529410 4051 ts_tablet_manager.cc:1939] T 64bd7ae7c2494015a2aef74ccc86ebb9 P cf25a8ecd16c47b995d657d5acfc10cb: Deleting consensus metadata
I20260812 03:51:34.529573 4052 ts_tablet_manager.cc:1939] T 64bd7ae7c2494015a2aef74ccc86ebb9 P 8a7e6b89c6f94bd7a6bb0e8985d2eeb9: Deleting consensus metadata
I20260812 03:51:34.529909 3167 catalog_manager.cc:4841] TS cf25a8ecd16c47b995d657d5acfc10cb (127.7.132.131:43585): tablet 64bd7ae7c2494015a2aef74ccc86ebb9 (table test-table [id=9850ed29dc264da38ee94a5c0d9a5b86]) successfully deleted
I20260812 03:51:34.530074 3167 catalog_manager.cc:4841] TS 8a7e6b89c6f94bd7a6bb0e8985d2eeb9 (127.7.132.130:44061): tablet 64bd7ae7c2494015a2aef74ccc86ebb9 (table test-table [id=9850ed29dc264da38ee94a5c0d9a5b86]) successfully deleted
W20260812 03:51:34.535364 3167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:37517 (error 108)
I20260812 03:51:34.535842 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:34.535731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:37517 (local address 127.0.0.1:39790)
0812 03:51:34.535779 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:34.535816 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:37517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 03:51:34.536032 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 3542
W20260812 03:51:34.536034 3167 catalog_manager.cc:4584] TS 66d7a5e895214a27bc51ebd5662da709 (127.7.132.129:37517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64bd7ae7c2494015a2aef74ccc86ebb9: Network error: Client connection negotiation failed: client connection to 127.7.132.129:37517: connect: Connection refused (error 111)
I20260812 03:51:34.552134 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 3769
I20260812 03:51:34.560887 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 3143
2026-08-12T03:51:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (816 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:34 dist-test-slave-39l8 krb5kdc[4063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:34 dist-test-slave-39l8 krb5kdc[4063](info): set up 2 sockets
Aug 12 03:51:34 dist-test-slave-39l8 krb5kdc[4063](info): commencing operation
krb5kdc: starting...
W20260812 03:51:35.724668 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:35 dist-test-slave-39l8 krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:51:35.832832 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44327
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:44327
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:35.908951 4080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:35.909121 4080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:35.909142 4080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:35.910241 4080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:35.910283 4080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:35.910297 4080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:35.910310 4080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:35.910323 4080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:35.912361 4080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:44327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:44327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:35.912588 4080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:35.912817 4080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:51:35.917091 4080 server_base.cc:1034] running on GCE node
W20260812 03:51:35.917069 4091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:35.917066 4089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:35.917069 4088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:35.917526 4080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:35.917817 4080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:35.918948 4080 hybrid_clock.cc:648] HybridClock initialized: now 1786506695918934 us; error 32 us; skew 500 ppm
Aug 12 03:51:35 dist-test-slave-39l8 krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506695, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:35.924633 4080 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:51:35.925046 4080 webserver.cc:460] Webserver started at http://127.7.132.190:33735/ using document root <none> and password file <none>
I20260812 03:51:35.925271 4080 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:35.925330 4080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:35.925464 4080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:35.926312 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "988d2dfc24dc45968efc509dfe6fe8f8"
format_stamp: "Formatted at 2026-08-12 03:51:35 on dist-test-slave-39l8"
server_key: "9d6e9d22ba76282918a5099385ec8e08"
server_key_iv: "3ba8b7feb275410e62dde701ee6918b0"
server_key_version: "encryptionkey@0"
I20260812 03:51:35.926667 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "988d2dfc24dc45968efc509dfe6fe8f8"
format_stamp: "Formatted at 2026-08-12 03:51:35 on dist-test-slave-39l8"
server_key: "9d6e9d22ba76282918a5099385ec8e08"
server_key_iv: "3ba8b7feb275410e62dde701ee6918b0"
server_key_version: "encryptionkey@0"
I20260812 03:51:35.927887 4080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:51:35.928545 4102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:35.928704 4080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:35.928798 4080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "988d2dfc24dc45968efc509dfe6fe8f8"
format_stamp: "Formatted at 2026-08-12 03:51:35 on dist-test-slave-39l8"
server_key: "9d6e9d22ba76282918a5099385ec8e08"
server_key_iv: "3ba8b7feb275410e62dde701ee6918b0"
server_key_version: "encryptionkey@0"
I20260812 03:51:35.928886 4080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:35.940925 4080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:35.942485 4080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:35.942631 4080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:35.946300 4080 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:44327
I20260812 03:51:35.946329 4193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:44327 every 8 connection(s)
I20260812 03:51:35.946612 4080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:35.947013 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 4080
I20260812 03:51:35.947127 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:35.947293 7698 external_mini_cluster.cc:1428] Setting key b744b708905c0203328f23b9afc6a422
I20260812 03:51:35.947521 4194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:35.949896 4194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8: Bootstrap starting.
I20260812 03:51:35.950536 4194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:35.950891 4194 log.cc:824] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:35.951592 4194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8: No bootstrap required, opened a new log
Aug 12 03:51:35 dist-test-slave-39l8 krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:35.953902 4194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988d2dfc24dc45968efc509dfe6fe8f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44327 } }
I20260812 03:51:35.954074 4194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:35.954120 4194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 988d2dfc24dc45968efc509dfe6fe8f8, State: Initialized, Role: FOLLOWER
I20260812 03:51:35.954257 4194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988d2dfc24dc45968efc509dfe6fe8f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44327 } }
I20260812 03:51:35.954340 4194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:35.954362 4194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:35.954422 4194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:35.955016 4194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988d2dfc24dc45968efc509dfe6fe8f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44327 } }
I20260812 03:51:35.955111 4194 leader_election.cc:302] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 988d2dfc24dc45968efc509dfe6fe8f8; no voters:
I20260812 03:51:35.955322 4194 leader_election.cc:288] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:35.955375 4200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:35.955518 4200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [term 1 LEADER]: Becoming Leader. State: Replica: 988d2dfc24dc45968efc509dfe6fe8f8, State: Running, Role: LEADER
I20260812 03:51:35.955585 4194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:35.955602 4200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988d2dfc24dc45968efc509dfe6fe8f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44327 } }
I20260812 03:51:35.955950 4200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 988d2dfc24dc45968efc509dfe6fe8f8. Latest consensus state: current_term: 1 leader_uuid: "988d2dfc24dc45968efc509dfe6fe8f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988d2dfc24dc45968efc509dfe6fe8f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44327 } } }
I20260812 03:51:35.956043 4200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:35.955957 4203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "988d2dfc24dc45968efc509dfe6fe8f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988d2dfc24dc45968efc509dfe6fe8f8" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 44327 } } }
I20260812 03:51:35.956233 4203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:35.956336 4214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:35.956841 4214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:35.957644 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:35.947718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39552 (local address 127.7.132.190:44327)
0812 03:51:35.947875 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:35.947884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:35.947911 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:51:35.948381 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:35.948387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:35.948423 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:35.948556 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:35.949123 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:35.949883 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:35.950654 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:35.951173 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:35.951291 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:35.953786 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:35.953790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:35.953790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:35.953799 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:35.955625 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:35.956040 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:35.956042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:35.956043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:35.956174 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:35.956437 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:35.956437 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:35.956438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:35.956722 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:35.957270 (+ 548us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:35.957336 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:51:35.957478 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260812 03:51:35.958427 4214 catalog_manager.cc:1347] Generated new cluster ID: 2a18ba86a323499daafd259a3d5f0ea6
I20260812 03:51:35.958479 4214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:35.989933 4214 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:35.990626 4214 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:35.995172 4214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8: Generated new TSK 0
I20260812 03:51:35.995339 4214 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:51:36.026921 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44327
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:36.103891 4231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:36.104081 4231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:36.104103 4231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:36.105211 4231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:36.105252 4231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:36.105304 4231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:51:36.106675 4231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:36.106918 4231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:36.107138 4231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:36.111985 4242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.111990 4238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.112075 4236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:36.112232 4231 server_base.cc:1034] running on GCE node
I20260812 03:51:36.112460 4231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:36.112720 4231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:36.113894 4231 hybrid_clock.cc:648] HybridClock initialized: now 1786506696113884 us; error 35 us; skew 500 ppm
Aug 12 03:51:36 dist-test-slave-39l8 krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506696, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:36.116372 4231 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:51:36.116721 4231 webserver.cc:460] Webserver started at http://127.7.132.129:38921/ using document root <none> and password file <none>
I20260812 03:51:36.116876 4231 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:36.116911 4231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:36.116987 4231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:36.117848 4231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "95d424abad984705ac60a5ed6a0cc382"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "52d1d81d3e1c7bb28ee756298285c5c1"
server_key_iv: "a50c676fdc98a29787e65c684e5d9294"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.118137 4231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "95d424abad984705ac60a5ed6a0cc382"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "52d1d81d3e1c7bb28ee756298285c5c1"
server_key_iv: "a50c676fdc98a29787e65c684e5d9294"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.119422 4231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:51:36.120107 4251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.120299 4231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:36.120385 4231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "95d424abad984705ac60a5ed6a0cc382"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "52d1d81d3e1c7bb28ee756298285c5c1"
server_key_iv: "a50c676fdc98a29787e65c684e5d9294"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.120440 4231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:36.134846 4231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:36.136333 4231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:36.136426 4231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:36.136566 4231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:36.136823 4231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:36.136853 4231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.136870 4231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:36.136911 4231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.142266 4231 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:33385
I20260812 03:51:36.142334 4445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:33385 every 8 connection(s)
I20260812 03:51:36.142627 4231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
Aug 12 03:51:36 dist-test-slave-39l8 krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506696, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:36.150481 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.143311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:36433 (local address 127.7.132.190:44327)
0812 03:51:36.143367 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.143370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.143559 (+ 189us) server_negotiation.cc:407] Connection header received
0812 03:51:36.143966 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.143967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.143975 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.144021 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:36.144795 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.145230 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.146064 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.146232 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.146276 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.148072 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:36.148073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:36.148073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:36.148079 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:36.149403 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:36.149723 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:36.149723 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:36.149724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:36.149771 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:36.149987 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:36.149988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:36.149989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:36.150087 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:36.150137 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.150350 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.150415 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:51:36.150635 4448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.143287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44327 (local address 127.7.132.129:36433)
0812 03:51:36.143451 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.143484 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.143891 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.144070 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.144073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.144546 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:36.144716 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.144731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.145307 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.145308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.145996 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.145998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.146298 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.146299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.146381 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.146777 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:36.146785 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:36.147990 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:36.149468 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:36.149473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:36.149475 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:36.149662 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:36.149824 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:36.149825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:36.149825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:36.149933 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:36.150142 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:36.150146 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:36.150246 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.150321 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.150431 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260812 03:51:36.151197 4446 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44327
I20260812 03:51:36.151291 4446 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:36.151459 4446 heartbeater.cc:511] Master 127.7.132.190:44327 requested a full tablet report, sending...
I20260812 03:51:36.151588 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 4231
I20260812 03:51:36.151710 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:51:36.151909 7698 external_mini_cluster.cc:1428] Setting key 78fbf23714365198a4cd7c03a8afefeb
I20260812 03:51:36.151961 4129 ts_manager.cc:194] Registered new tserver with Master: 95d424abad984705ac60a5ed6a0cc382 (127.7.132.129:33385)
I20260812 03:51:36.152916 4129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:36433
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:51:36.175279 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44327
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:36.251971 4453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:36.252177 4453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:36.252198 4453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:36.253218 4453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:36.253249 4453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:36.253273 4453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:51:36.254549 4453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:36.254752 4453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:36.254972 4453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:36.259572 4462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.259569 4463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:36.259686 4453 server_base.cc:1034] running on GCE node
W20260812 03:51:36.259640 4466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:36.259958 4453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:36.260150 4453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:36.261281 4453 hybrid_clock.cc:648] HybridClock initialized: now 1786506696261271 us; error 29 us; skew 500 ppm
Aug 12 03:51:36 dist-test-slave-39l8 krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506696, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:36.263649 4453 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:51:36.263983 4453 webserver.cc:460] Webserver started at http://127.7.132.130:35615/ using document root <none> and password file <none>
I20260812 03:51:36.264137 4453 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:36.264179 4453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:36.264259 4453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:36.265117 4453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "03a743b0ab3444908ef8a83b4ab22d90"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "b64aecab0292cec3cc1d745279b8bd6e"
server_key_iv: "5c8d96ddc265c70839aa872345e3592f"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.265407 4453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "03a743b0ab3444908ef8a83b4ab22d90"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "b64aecab0292cec3cc1d745279b8bd6e"
server_key_iv: "5c8d96ddc265c70839aa872345e3592f"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.266470 4453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:51:36.267180 4484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.267422 4453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:51:36.267504 4453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "03a743b0ab3444908ef8a83b4ab22d90"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "b64aecab0292cec3cc1d745279b8bd6e"
server_key_iv: "5c8d96ddc265c70839aa872345e3592f"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.267553 4453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:36.279944 4453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:36.281661 4453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:36.281756 4453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:36.281904 4453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:36.282119 4453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:36.282145 4453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.282163 4453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:36.282174 4453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.287490 4453 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:42635
I20260812 03:51:36.287539 4666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:42635 every 8 connection(s)
I20260812 03:51:36.287786 4453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:51:36.289481 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 4453
I20260812 03:51:36.289590 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:51:36.289773 7698 external_mini_cluster.cc:1428] Setting key 9c60c68128b8e4e9e6375e7853929744
Aug 12 03:51:36 dist-test-slave-39l8 krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506696, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:36.297158 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.288426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:48773 (local address 127.7.132.190:44327)
0812 03:51:36.288467 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.288468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.288634 (+ 166us) server_negotiation.cc:407] Connection header received
0812 03:51:36.289094 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.289096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.289109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.289163 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:36.290222 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.290894 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.291814 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.292112 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.292195 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.294423 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:36.294425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:36.294425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:36.294432 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:36.295766 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:36.296243 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:36.296243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:36.296244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:36.296292 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:36.296557 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:36.296557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:36.296558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:36.296652 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:36.296698 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.297028 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.297088 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:51:36.297367 4669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.288382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44327 (local address 127.7.132.130:48773)
0812 03:51:36.288531 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.288561 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.288985 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.289220 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.289223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.289850 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:36.290128 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.290133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.290997 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.290998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.291725 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.291727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.292251 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.292252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.292364 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.292841 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:36.292850 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:36.294327 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:36.295846 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:36.295853 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:36.295855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:36.296166 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:36.296355 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:36.296356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:36.296356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:36.296491 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:36.296713 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:36.296720 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:36.296862 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.297001 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.297162 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260812 03:51:36.297806 4667 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44327
I20260812 03:51:36.297904 4667 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:36.298059 4667 heartbeater.cc:511] Master 127.7.132.190:44327 requested a full tablet report, sending...
I20260812 03:51:36.298418 4129 ts_manager.cc:194] Registered new tserver with Master: 03a743b0ab3444908ef8a83b4ab22d90 (127.7.132.130:42635)
I20260812 03:51:36.298808 4129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:48773
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:51:36.314914 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44327
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:36.390823 4678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:36.391008 4678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:36.391031 4678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:36.392032 4678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:36.392086 4678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:36.392109 4678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:51:36.393500 4678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:44327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:36.393725 4678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:36.393926 4678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:36.398109 4690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.398146 4686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.398109 4684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:36.398339 4678 server_base.cc:1034] running on GCE node
I20260812 03:51:36.398551 4678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:36.398811 4678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:36.399969 4678 hybrid_clock.cc:648] HybridClock initialized: now 1786506696399958 us; error 26 us; skew 500 ppm
Aug 12 03:51:36 dist-test-slave-39l8 krb5kdc[4063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506696, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:36.402354 4678 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:51:36.402822 4678 webserver.cc:460] Webserver started at http://127.7.132.131:38567/ using document root <none> and password file <none>
I20260812 03:51:36.403051 4678 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:36.403092 4678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:36.403216 4678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:36.404076 4678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "0056eefc1c04419296c099caacb8a8b6"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "27e88afc7cf80d7451e6f76be7ee90e9"
server_key_iv: "9875779d185425df68bafb663e1dac57"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.404408 4678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "0056eefc1c04419296c099caacb8a8b6"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "27e88afc7cf80d7451e6f76be7ee90e9"
server_key_iv: "9875779d185425df68bafb663e1dac57"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.405709 4678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:51:36.406417 4705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.406592 4678 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:36.406674 4678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "0056eefc1c04419296c099caacb8a8b6"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "27e88afc7cf80d7451e6f76be7ee90e9"
server_key_iv: "9875779d185425df68bafb663e1dac57"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.406759 4678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:36.426304 4678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:36.427974 4678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:36.428102 4678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:36.428309 4678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:36.428630 4678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:36.428668 4678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.428689 4678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:36.428728 4678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.434442 4678 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:41549
I20260812 03:51:36.434481 4888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:41549 every 8 connection(s)
I20260812 03:51:36.434791 4678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:51:36.439137 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 4678
I20260812 03:51:36.439253 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:51:36.439458 7698 external_mini_cluster.cc:1428] Setting key 0dc2a0d656d2275e7bccdd41cdc4bac3
Aug 12 03:51:36 dist-test-slave-39l8 krb5kdc[4063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506696, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:36.443656 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.435493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:54497 (local address 127.7.132.190:44327)
0812 03:51:36.435539 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.435541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.435701 (+ 160us) server_negotiation.cc:407] Connection header received
0812 03:51:36.436132 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.436133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.436141 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.436192 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:36.436966 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.437420 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.438249 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.438426 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.438471 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.441054 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:36.441056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:36.441056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:36.441065 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:36.442539 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:36.442942 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:36.442943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:36.442944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:36.442991 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:36.443187 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:36.443187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:36.443187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:36.443279 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:36.443325 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.443540 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.443598 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:51:36.443845 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.435458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:44327 (local address 127.7.132.131:54497)
0812 03:51:36.435601 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.435638 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.436055 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.436239 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.436242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.436734 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:36.436898 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.436902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.437497 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.437498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.438185 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.438187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.438493 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.438493 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.438584 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.439025 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:36.439036 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:36.440927 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:36.442608 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:36.442614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:36.442615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:36.442883 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:36.443048 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:36.443049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:36.443049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:36.443141 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:36.443335 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:36.443338 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:36.443451 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.443526 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.443634 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260812 03:51:36.444321 4889 heartbeater.cc:347] Connected to a master server at 127.7.132.190:44327
I20260812 03:51:36.444427 4889 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:36.444600 4889 heartbeater.cc:511] Master 127.7.132.190:44327 requested a full tablet report, sending...
I20260812 03:51:36.444960 4129 ts_manager.cc:194] Registered new tserver with Master: 0056eefc1c04419296c099caacb8a8b6 (127.7.132.131:41549)
I20260812 03:51:36.445308 4129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:54497
I20260812 03:51:36.452337 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:51:36.459538 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.453700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.7.132.190:44327)
0812 03:51:36.453763 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.453765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.453863 (+ 98us) server_negotiation.cc:407] Connection header received
0812 03:51:36.453900 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.453900 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.453908 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.453955 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:36.454473 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.454974 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.455725 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.455895 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.455947 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.457254 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:36.457255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:36.457255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:36.457261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:36.458548 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:36.458926 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:36.458927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:36.458927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:36.458977 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:36.459138 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:36.459139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:36.459139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:36.459241 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:36.459327 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.459426 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.459476 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:51:36.461190 4129 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:51:36.462786 4129 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:51:36.468438 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.464893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:42635 (local address 127.0.0.1:56572)
0812 03:51:36.465171 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.465181 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.465236 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.465482 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.465484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.465486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:36.465529 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.465531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.466696 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.466696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.467419 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.467421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.468171 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.468172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.468246 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.468258 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.468324 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.468373 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":1,"thread_start_us":34,"threads_started":1}
I20260812 03:51:36.468446 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.465007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56572 (local address 127.7.132.130:42635)
0812 03:51:36.465157 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.465162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.465246 (+ 84us) server_negotiation.cc:407] Connection header received
0812 03:51:36.465303 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.465305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.465320 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.465403 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:36.465642 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.466606 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.467514 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.468090 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.468170 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.468225 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.468347 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.468382 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260812 03:51:36.468643 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.464707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:33385 (local address 127.0.0.1:36866)
0812 03:51:36.464853 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.464890 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.464946 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.465418 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.465418 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.465420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:36.465453 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.465454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.466373 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.466375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.467595 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.467596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.468411 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.468412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.468476 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.468484 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.468512 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.468555 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":7,"thread_start_us":48,"threads_started":1}
I20260812 03:51:36.468644 4910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.464733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:41549 (local address 127.0.0.1:39644)
0812 03:51:36.464969 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.464978 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.465024 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.465331 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.465333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.465335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:36.465414 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.465416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.466557 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.466558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.467560 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.467563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.468411 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.468412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.468476 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.468482 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.468512 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.468555 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":31,"thread_start_us":48,"threads_started":1}
I20260812 03:51:36.468892 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.464989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39644 (local address 127.7.132.131:41549)
0812 03:51:36.465156 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.465160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.465170 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:51:36.465208 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.465210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.465224 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.465317 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:36.465494 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.466460 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.467666 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.468310 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.468693 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.468755 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.468782 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.468818 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
I20260812 03:51:36.468997 4909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.464823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.7.132.129:33385)
0812 03:51:36.465204 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.465212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.465238 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:51:36.465277 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.465279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.465293 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.465417 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:36.465493 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.466274 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.467675 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.468310 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.468802 (+ 492us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.468856 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.468878 (+ 22us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.468920 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":77,"threads_started":1}
I20260812 03:51:36.469453 4779 tablet_service.cc:1467] Processing CreateTablet for tablet 0a773c827e24489aba9d6d421804125a (DEFAULT_TABLE table=test-table [id=5e44e4c25cbd4e599c825a3c5cc5bcce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:36.469448 4562 tablet_service.cc:1467] Processing CreateTablet for tablet 0a773c827e24489aba9d6d421804125a (DEFAULT_TABLE table=test-table [id=5e44e4c25cbd4e599c825a3c5cc5bcce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:36.469725 4779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a773c827e24489aba9d6d421804125a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:36.469784 4562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a773c827e24489aba9d6d421804125a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:36.472388 4916 tablet_bootstrap.cc:492] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: Bootstrap starting.
I20260812 03:51:36.472388 4915 tablet_bootstrap.cc:492] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Bootstrap starting.
I20260812 03:51:36.472870 4336 tablet_service.cc:1467] Processing CreateTablet for tablet 0a773c827e24489aba9d6d421804125a (DEFAULT_TABLE table=test-table [id=5e44e4c25cbd4e599c825a3c5cc5bcce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:36.473008 4916 tablet_bootstrap.cc:654] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:36.473145 4915 tablet_bootstrap.cc:654] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:36.473214 4336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a773c827e24489aba9d6d421804125a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:36.473562 4916 log.cc:824] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:36.473713 4915 log.cc:824] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:36.474280 4916 tablet_bootstrap.cc:492] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: No bootstrap required, opened a new log
I20260812 03:51:36.474366 4916 ts_tablet_manager.cc:1397] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:36.474454 4915 tablet_bootstrap.cc:492] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: No bootstrap required, opened a new log
I20260812 03:51:36.474527 4915 ts_tablet_manager.cc:1397] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:36.475845 4916 raft_consensus.cc:348] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.476022 4916 raft_consensus.cc:374] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:36.476066 4916 raft_consensus.cc:729] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0056eefc1c04419296c099caacb8a8b6, State: Initialized, Role: FOLLOWER
I20260812 03:51:36.476178 4916 consensus_queue.cc:260] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.476461 4916 ts_tablet_manager.cc:1428] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:36.476488 4889 heartbeater.cc:503] Master 127.7.132.190:44327 was elected leader, sending a full tablet report...
I20260812 03:51:36.476596 4915 raft_consensus.cc:348] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.476771 4915 raft_consensus.cc:374] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:36.476835 4915 raft_consensus.cc:729] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a743b0ab3444908ef8a83b4ab22d90, State: Initialized, Role: FOLLOWER
I20260812 03:51:36.476940 4915 consensus_queue.cc:260] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.477214 4915 ts_tablet_manager.cc:1428] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:51:36.477233 4667 heartbeater.cc:503] Master 127.7.132.190:44327 was elected leader, sending a full tablet report...
I20260812 03:51:36.477391 4922 tablet_bootstrap.cc:492] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382: Bootstrap starting.
I20260812 03:51:36.477977 4922 tablet_bootstrap.cc:654] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:36.478461 4922 log.cc:824] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:36.479172 4922 tablet_bootstrap.cc:492] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382: No bootstrap required, opened a new log
I20260812 03:51:36.479254 4922 ts_tablet_manager.cc:1397] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:36.480684 4922 raft_consensus.cc:348] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.480859 4922 raft_consensus.cc:374] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:36.480907 4922 raft_consensus.cc:729] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d424abad984705ac60a5ed6a0cc382, State: Initialized, Role: FOLLOWER
I20260812 03:51:36.481016 4922 consensus_queue.cc:260] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.481299 4922 ts_tablet_manager.cc:1428] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:36.481374 4446 heartbeater.cc:503] Master 127.7.132.190:44327 was elected leader, sending a full tablet report...
W20260812 03:51:36.538337 4668 tablet.cc:2369] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:51:36.545126 4919 raft_consensus.cc:482] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:36.545343 4919 raft_consensus.cc:504] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.545723 4919 leader_election.cc:288] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95d424abad984705ac60a5ed6a0cc382 (127.7.132.129:33385), 03a743b0ab3444908ef8a83b4ab22d90 (127.7.132.130:42635)
I20260812 03:51:36.548097 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.545858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:42635 (local address 127.7.132.131:41625)
0812 03:51:36.545894 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.545904 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.545946 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.546117 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.546118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.546120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:36.546186 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.546187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.546768 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.546769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.547380 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.547382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.547870 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.547871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.547939 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.547942 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.547999 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.548037 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260812 03:51:36.548096 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.545952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41625 (local address 127.7.132.130:42635)
0812 03:51:36.546006 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.546008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.546016 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:36.546041 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.546042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.546051 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.546100 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:36.546245 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.546663 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.547464 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.547792 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.547870 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.547879 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.548004 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.548038 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:51:36.548346 4909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.545950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:37477 (local address 127.7.132.129:33385)
0812 03:51:36.545997 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.545998 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.546117 (+ 119us) server_negotiation.cc:407] Connection header received
0812 03:51:36.546175 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.546176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.546186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.546230 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:36.546367 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.546827 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.547636 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.547969 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.548046 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.548054 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.548220 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.548289 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:51:36.548447 4599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a773c827e24489aba9d6d421804125a" candidate_uuid: "0056eefc1c04419296c099caacb8a8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a743b0ab3444908ef8a83b4ab22d90" is_pre_election: true
I20260812 03:51:36.548548 4929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.545867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:33385 (local address 127.7.132.131:37477)
0812 03:51:36.546056 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.546064 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.546119 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.546233 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.546234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.546235 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:36.546299 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.546300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.546899 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.546900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.547547 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.547549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.548049 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.548050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.548110 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.548114 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.548421 (+ 307us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.548466 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":17,"thread_start_us":116,"threads_started":1}
I20260812 03:51:36.548609 4599 raft_consensus.cc:2454] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0056eefc1c04419296c099caacb8a8b6 in term 0.
I20260812 03:51:36.548837 4711 leader_election.cc:302] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0056eefc1c04419296c099caacb8a8b6, 03a743b0ab3444908ef8a83b4ab22d90; no voters:
I20260812 03:51:36.548964 4919 raft_consensus.cc:2781] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:51:36.549013 4919 raft_consensus.cc:482] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:51:36.548971 4373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a773c827e24489aba9d6d421804125a" candidate_uuid: "0056eefc1c04419296c099caacb8a8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d424abad984705ac60a5ed6a0cc382" is_pre_election: true
I20260812 03:51:36.549052 4919 raft_consensus.cc:3037] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:36.549115 4373 raft_consensus.cc:2454] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0056eefc1c04419296c099caacb8a8b6 in term 0.
I20260812 03:51:36.549788 4919 raft_consensus.cc:504] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.549952 4919 leader_election.cc:288] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [CANDIDATE]: Term 1 election: Requested vote from peers 95d424abad984705ac60a5ed6a0cc382 (127.7.132.129:33385), 03a743b0ab3444908ef8a83b4ab22d90 (127.7.132.130:42635)
I20260812 03:51:36.550119 4373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a773c827e24489aba9d6d421804125a" candidate_uuid: "0056eefc1c04419296c099caacb8a8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95d424abad984705ac60a5ed6a0cc382"
I20260812 03:51:36.550132 4599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a773c827e24489aba9d6d421804125a" candidate_uuid: "0056eefc1c04419296c099caacb8a8b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a743b0ab3444908ef8a83b4ab22d90"
I20260812 03:51:36.550218 4373 raft_consensus.cc:3037] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:36.550215 4599 raft_consensus.cc:3037] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:36.550938 4599 raft_consensus.cc:2454] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0056eefc1c04419296c099caacb8a8b6 in term 1.
I20260812 03:51:36.551115 4711 leader_election.cc:302] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0056eefc1c04419296c099caacb8a8b6, 03a743b0ab3444908ef8a83b4ab22d90; no voters:
I20260812 03:51:36.551199 4373 raft_consensus.cc:2454] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0056eefc1c04419296c099caacb8a8b6 in term 1.
I20260812 03:51:36.551222 4919 raft_consensus.cc:2781] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:36.551350 4919 raft_consensus.cc:686] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 1 LEADER]: Becoming Leader. State: Replica: 0056eefc1c04419296c099caacb8a8b6, State: Running, Role: LEADER
I20260812 03:51:36.551450 4919 consensus_queue.cc:237] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } }
I20260812 03:51:36.552167 4129 catalog_manager.cc:5495] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0056eefc1c04419296c099caacb8a8b6 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "0056eefc1c04419296c099caacb8a8b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0056eefc1c04419296c099caacb8a8b6" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 41549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 } health_report { overall_health: UNKNOWN } } }
I20260812 03:51:36.564260 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.562136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39652 (local address 127.7.132.131:41549)
0812 03:51:36.562212 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.562214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.562222 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:36.562272 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.562273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.562290 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.562351 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:51:36.562490 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.562935 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.563652 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.563815 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.563862 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.564002 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:51:36.564077 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:51:36.564142 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.564195 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.564203 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:51:36.565889 4599 raft_consensus.cc:1261] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 1 FOLLOWER]: Refusing update from remote peer 0056eefc1c04419296c099caacb8a8b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:36.565889 4373 raft_consensus.cc:1261] T 0a773c827e24489aba9d6d421804125a P 95d424abad984705ac60a5ed6a0cc382 [term 1 FOLLOWER]: Refusing update from remote peer 0056eefc1c04419296c099caacb8a8b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:36.566267 4930 consensus_queue.cc:1035] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a743b0ab3444908ef8a83b4ab22d90" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 42635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:36.566406 4919 consensus_queue.cc:1035] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95d424abad984705ac60a5ed6a0cc382" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 33385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:36.568691 4935 mvcc.cc:205] Tried to move back new op lower bound from 7317531429131534336 to 7317531429075423232. Current Snapshot: MvccSnapshot[applied={T|T < 7317531429131534336}]
I20260812 03:51:36.569878 4937 mvcc.cc:205] Tried to move back new op lower bound from 7317531429131534336 to 7317531429075423232. Current Snapshot: MvccSnapshot[applied={T|T < 7317531429131534336 or (T in {7317531429131534336})}]
I20260812 03:51:36.571022 4129 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:36.571127 4129 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:36.571825 4129 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 988d2dfc24dc45968efc509dfe6fe8f8: Sending DeleteTablet for 3 replicas of tablet 0a773c827e24489aba9d6d421804125a
I20260812 03:51:36.572232 4779 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a773c827e24489aba9d6d421804125a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:36 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:39644
I20260812 03:51:36.572234 4336 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a773c827e24489aba9d6d421804125a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:36 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36866
I20260812 03:51:36.572458 4949 tablet_replica.cc:330] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: stopping tablet replica
I20260812 03:51:36.572515 4562 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a773c827e24489aba9d6d421804125a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:36 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:56572
I20260812 03:51:36.572567 4949 raft_consensus.cc:2229] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:51:36.572671 4949 raft_consensus.cc:2258] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:36.572748 4954 tablet_replica.cc:330] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: stopping tablet replica
I20260812 03:51:36.572875 4954 raft_consensus.cc:2229] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:51:36.573035 4954 raft_consensus.cc:2258] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:36.573038 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 4231
I20260812 03:51:36.573065 4949 ts_tablet_manager.cc:1905] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:36.575803 4949 ts_tablet_manager.cc:1918] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:36.575997 4949 log.cc:1196] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/0a773c827e24489aba9d6d421804125a
I20260812 03:51:36.576301 4949 ts_tablet_manager.cc:1939] T 0a773c827e24489aba9d6d421804125a P 0056eefc1c04419296c099caacb8a8b6: Deleting consensus metadata
I20260812 03:51:36.577785 4954 ts_tablet_manager.cc:1905] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:36.579267 4109 catalog_manager.cc:4841] TS 0056eefc1c04419296c099caacb8a8b6 (127.7.132.131:41549): tablet 0a773c827e24489aba9d6d421804125a (table test-table [id=5e44e4c25cbd4e599c825a3c5cc5bcce]) successfully deleted
I20260812 03:51:36.579282 4954 ts_tablet_manager.cc:1918] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:36.579365 4954 log.cc:1196] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/0a773c827e24489aba9d6d421804125a
I20260812 03:51:36.579674 4954 ts_tablet_manager.cc:1939] T 0a773c827e24489aba9d6d421804125a P 03a743b0ab3444908ef8a83b4ab22d90: Deleting consensus metadata
I20260812 03:51:36.580101 4109 catalog_manager.cc:4841] TS 03a743b0ab3444908ef8a83b4ab22d90 (127.7.132.130:42635): tablet 0a773c827e24489aba9d6d421804125a (table test-table [id=5e44e4c25cbd4e599c825a3c5cc5bcce]) successfully deleted
W20260812 03:51:36.585335 4107 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:33385 (error 108)
I20260812 03:51:36.585831 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.585681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:33385 (local address 127.0.0.1:36880)
0812 03:51:36.585755 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.585793 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:33385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 03:51:36.585925 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 4453
W20260812 03:51:36.585986 4107 catalog_manager.cc:4584] TS 95d424abad984705ac60a5ed6a0cc382 (127.7.132.129:33385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a773c827e24489aba9d6d421804125a: Network error: Client connection negotiation failed: client connection to 127.7.132.129:33385: connect: Connection refused (error 111)
I20260812 03:51:36.613976 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 4678
I20260812 03:51:36.623291 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 4080
2026-08-12T03:51:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2063 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T03:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:36Z Disabled control of system clock
I20260812 03:51:36.656044 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:45487
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:45487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:36.731182 4961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:36.731379 4961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:36.731395 4961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:36.732470 4961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:36.732508 4961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:36.732520 4961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:36.732532 4961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:36.732542 4961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:36.733770 4961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:45487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:45487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:36.733992 4961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:36.734275 4961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:36.738647 4972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.738639 4970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.738619 4969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:36.738911 4961 server_base.cc:1034] running on GCE node
I20260812 03:51:36.739291 4961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:36.739656 4961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:36.740824 4961 hybrid_clock.cc:648] HybridClock initialized: now 1786506696740810 us; error 44 us; skew 500 ppm
I20260812 03:51:36.741983 4961 webserver.cc:460] Webserver started at http://127.7.132.190:41997/ using document root <none> and password file <none>
I20260812 03:51:36.742244 4961 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:36.742307 4961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:36.742460 4961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:36.743526 4961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "5427ad59d80e4260bd25fc640dffc594"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "04f182d627c837a38945fbc7d2ca491f"
server_key_iv: "e33195e0e2fa8eaed4110f094a76039a"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.743865 4961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "5427ad59d80e4260bd25fc640dffc594"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "04f182d627c837a38945fbc7d2ca491f"
server_key_iv: "e33195e0e2fa8eaed4110f094a76039a"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.745124 4961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:51:36.745798 4982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.745952 4961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:36.746040 4961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "5427ad59d80e4260bd25fc640dffc594"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "04f182d627c837a38945fbc7d2ca491f"
server_key_iv: "e33195e0e2fa8eaed4110f094a76039a"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.746161 4961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:36.757180 4961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:36.757419 4961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:36.757546 4961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:36.761438 4961 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:45487
I20260812 03:51:36.761467 5068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:45487 every 8 connection(s)
I20260812 03:51:36.761754 4961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:36.762743 5069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:36.764741 5069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594: Bootstrap starting.
I20260812 03:51:36.765270 5069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:36.765563 5069 log.cc:824] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:36.766170 5069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594: No bootstrap required, opened a new log
I20260812 03:51:36.767742 5069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5427ad59d80e4260bd25fc640dffc594" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45487 } }
I20260812 03:51:36.767867 5069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:36.767887 5069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5427ad59d80e4260bd25fc640dffc594, State: Initialized, Role: FOLLOWER
I20260812 03:51:36.768069 5069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5427ad59d80e4260bd25fc640dffc594" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45487 } }
I20260812 03:51:36.768149 5069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:36.768198 5069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:36.768244 5069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:36.768749 5069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5427ad59d80e4260bd25fc640dffc594" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45487 } }
I20260812 03:51:36.768857 5069 leader_election.cc:302] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5427ad59d80e4260bd25fc640dffc594; no voters:
I20260812 03:51:36.769019 5069 leader_election.cc:288] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:36.769138 5073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:36.769280 5069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:36.769315 5073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [term 1 LEADER]: Becoming Leader. State: Replica: 5427ad59d80e4260bd25fc640dffc594, State: Running, Role: LEADER
I20260812 03:51:36.769428 5073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5427ad59d80e4260bd25fc640dffc594" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45487 } }
I20260812 03:51:36.769893 5077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5427ad59d80e4260bd25fc640dffc594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5427ad59d80e4260bd25fc640dffc594" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45487 } } }
I20260812 03:51:36.769999 5077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:36.770017 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 4961
I20260812 03:51:36.770126 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:36.770344 7698 external_mini_cluster.cc:1428] Setting key 2edba8fc0de21d89a36fd1edf8e06335
I20260812 03:51:36.770524 5078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5427ad59d80e4260bd25fc640dffc594. Latest consensus state: current_term: 1 leader_uuid: "5427ad59d80e4260bd25fc640dffc594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5427ad59d80e4260bd25fc640dffc594" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 45487 } } }
I20260812 03:51:36.770630 5078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594 [sys.catalog]: This master's current role is: LEADER
W20260812 03:51:36.771195 5108 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:51:36.771252 5108 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:51:36.771320 5091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:36.771531 5091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:36.773598 5091 catalog_manager.cc:1347] Generated new cluster ID: 649a72fd3eea4e3ab81d971597fd8231
I20260812 03:51:36.773650 5091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:36.776101 5114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.770972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54170 (local address 127.7.132.190:45487)
0812 03:51:36.771184 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.771191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.771220 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:51:36.771329 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.771335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.771405 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.771535 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:36.771828 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.773157 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.774085 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.774650 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.774796 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.775418 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:36.775422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:36.775422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:36.775625 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:36.775664 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:36.775678 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:36.775749 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.775828 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.775901 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":114,"threads_started":1}
I20260812 03:51:36.779657 5091 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:36.780117 5091 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:36.786491 5091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594: Generated new TSK 0
I20260812 03:51:36.786636 5091 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:36.791404 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45487
--builtin_ntp_servers=127.7.132.148:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:36.868511 5117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:36.868696 5117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:36.868716 5117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:36.869943 5117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:36.870012 5117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:36.870064 5117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:51:36.871567 5117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:36.871798 5117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:36.872105 5117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:36.877593 5133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.877650 5128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.877825 5127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:36.878094 5117 server_base.cc:1034] running on GCE node
I20260812 03:51:36.878296 5117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:36.878552 5117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:36.879729 5117 hybrid_clock.cc:648] HybridClock initialized: now 1786506696879688 us; error 58 us; skew 500 ppm
I20260812 03:51:36.881035 5117 webserver.cc:460] Webserver started at http://127.7.132.129:34283/ using document root <none> and password file <none>
I20260812 03:51:36.881279 5117 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:36.881346 5117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:36.881479 5117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:36.882462 5117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "c02bc416e18348d1994c7551f39eb4bf"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "e8de22290834b5c1e9f8200aa13791b0"
server_key_iv: "f3315ee061591ec91f47a98273005273"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.882861 5117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "c02bc416e18348d1994c7551f39eb4bf"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "e8de22290834b5c1e9f8200aa13791b0"
server_key_iv: "f3315ee061591ec91f47a98273005273"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.884213 5117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:51:36.884946 5143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.885095 5117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:36.885195 5117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "c02bc416e18348d1994c7551f39eb4bf"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "e8de22290834b5c1e9f8200aa13791b0"
server_key_iv: "f3315ee061591ec91f47a98273005273"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.885267 5117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:36.896517 5117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:36.896836 5117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:36.896963 5117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:36.897181 5117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:36.897460 5117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:36.897491 5117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.897509 5117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:36.897558 5117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.902896 5117 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:40367
I20260812 03:51:36.902948 5321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:40367 every 8 connection(s)
I20260812 03:51:36.903216 5117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:51:36.906920 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 5117
I20260812 03:51:36.907037 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:51:36.907294 7698 external_mini_cluster.cc:1428] Setting key c2f40803221e9febc3d20a208b1dbb9a
I20260812 03:51:36.908267 5114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.903830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:34447 (local address 127.7.132.190:45487)
0812 03:51:36.903869 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:36.903871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:36.904147 (+ 276us) server_negotiation.cc:407] Connection header received
0812 03:51:36.904637 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:36.904638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:36.904653 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:36.904710 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:36.905354 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.905814 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.906824 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.907006 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.907112 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.908016 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:36.908017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:36.908017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:36.908022 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:36.908034 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:36.908038 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:36.908087 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:36.908171 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:51:36.908198 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:51:36.908330 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45487
--builtin_ntp_servers=127.7.132.148:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:51:36.908458 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:36.903832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45487 (local address 127.7.132.129:34447)
0812 03:51:36.904015 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:36.904068 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:36.904526 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:36.904775 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:36.904778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:36.904787 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:36.905260 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.905264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.905907 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.905908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.906629 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:36.906631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:36.907123 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:36.907124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:36.907305 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:36.907843 (+ 538us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:36.907843 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:36.907878 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:36.907905 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:36.908095 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:36.908105 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:36.908108 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:36.908171 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 03:51:36.908261 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260812 03:51:36.908916 5323 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45487
I20260812 03:51:36.909008 5323 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:36.909183 5323 heartbeater.cc:511] Master 127.7.132.190:45487 requested a full tablet report, sending...
I20260812 03:51:36.909641 5010 ts_manager.cc:194] Registered new tserver with Master: c02bc416e18348d1994c7551f39eb4bf (127.7.132.129:40367)
I20260812 03:51:36.910518 5010 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.129:34447
W20260812 03:51:36.984283 5327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:36.984444 5327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:36.984462 5327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:36.985500 5327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:36.985536 5327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:36.985559 5327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:51:36.986790 5327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:36.987003 5327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:36.987279 5327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:36.991524 5338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.991511 5336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:36.991511 5341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:36.991919 5327 server_base.cc:1034] running on GCE node
I20260812 03:51:36.992097 5327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:36.992272 5327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:36.993410 5327 hybrid_clock.cc:648] HybridClock initialized: now 1786506696993388 us; error 37 us; skew 500 ppm
I20260812 03:51:36.994428 5327 webserver.cc:460] Webserver started at http://127.7.132.130:35333/ using document root <none> and password file <none>
I20260812 03:51:36.994626 5327 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:36.994666 5327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:36.994815 5327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:36.995642 5327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "421bdc5b917d4df780572a264549542d"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "9e24ec35abccacef198b5fd84d644a9f"
server_key_iv: "7be46c3fe0ae0a1092de97a6a37b5bb6"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.995903 5327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "421bdc5b917d4df780572a264549542d"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "9e24ec35abccacef198b5fd84d644a9f"
server_key_iv: "7be46c3fe0ae0a1092de97a6a37b5bb6"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.996958 5327 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:51:36.997540 5352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:36.997685 5327 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:36.997776 5327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "421bdc5b917d4df780572a264549542d"
format_stamp: "Formatted at 2026-08-12 03:51:36 on dist-test-slave-39l8"
server_key: "9e24ec35abccacef198b5fd84d644a9f"
server_key_iv: "7be46c3fe0ae0a1092de97a6a37b5bb6"
server_key_version: "encryptionkey@0"
I20260812 03:51:36.997848 5327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:37.021831 5327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:37.022076 5327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:37.022210 5327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:37.022449 5327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:37.022815 5327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:37.022869 5327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:37.022898 5327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:37.022921 5327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:37.027797 5327 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37549
I20260812 03:51:37.027876 5538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37549 every 8 connection(s)
I20260812 03:51:37.028137 5327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:51:37.032464 5114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.028749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:33127 (local address 127.7.132.190:45487)
0812 03:51:37.028796 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.028798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.029011 (+ 213us) server_negotiation.cc:407] Connection header received
0812 03:51:37.029391 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.029392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.029402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.029442 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:37.029972 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.030426 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.031354 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.031518 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.031572 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.032269 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:37.032270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:37.032270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:37.032274 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:37.032283 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:37.032286 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:37.032320 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.032394 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.032413 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:51:37.032585 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.028735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45487 (local address 127.7.132.130:33127)
0812 03:51:37.028888 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:37.028930 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:37.029316 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:37.029493 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:37.029496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:37.029503 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:37.029895 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.029898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.030503 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.030504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.031279 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.031281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.031592 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.031593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.031680 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.032157 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:37.032157 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:37.032181 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:37.032199 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:37.032322 (+ 123us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:37.032324 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:37.032326 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:51:37.032366 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 03:51:37.032428 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260812 03:51:37.033046 5540 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45487
I20260812 03:51:37.033134 5540 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:37.033305 5540 heartbeater.cc:511] Master 127.7.132.190:45487 requested a full tablet report, sending...
I20260812 03:51:37.033646 5010 ts_manager.cc:194] Registered new tserver with Master: 421bdc5b917d4df780572a264549542d (127.7.132.130:37549)
I20260812 03:51:37.033926 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 5327
I20260812 03:51:37.033960 5010 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.130:33127
I20260812 03:51:37.034024 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:51:37.034233 7698 external_mini_cluster.cc:1428] Setting key b40ec61f81e686c533a175f2674e60b5
I20260812 03:51:37.034981 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45487
--builtin_ntp_servers=127.7.132.148:32945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:37.111246 5544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:37.111449 5544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:37.111481 5544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:37.112602 5544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:37.112665 5544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:37.112707 5544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:51:37.113927 5544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:32945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:45487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:37.114153 5544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:37.114439 5544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:37.119004 5555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:37.118993 5556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:37.118992 5559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:37.119274 5544 server_base.cc:1034] running on GCE node
I20260812 03:51:37.119457 5544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:37.119633 5544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:37.120795 5544 hybrid_clock.cc:648] HybridClock initialized: now 1786506697120787 us; error 27 us; skew 500 ppm
I20260812 03:51:37.121959 5544 webserver.cc:460] Webserver started at http://127.7.132.131:45189/ using document root <none> and password file <none>
I20260812 03:51:37.122189 5544 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:37.122264 5544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:37.122414 5544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:37.123327 5544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "2567baa83b80447d9199edae59f47274"
format_stamp: "Formatted at 2026-08-12 03:51:37 on dist-test-slave-39l8"
server_key: "84b3b701d12e6009b0caad0ea2d34163"
server_key_iv: "cd7585e88c622dc069abdc653f68bc7d"
server_key_version: "encryptionkey@0"
I20260812 03:51:37.123656 5544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "2567baa83b80447d9199edae59f47274"
format_stamp: "Formatted at 2026-08-12 03:51:37 on dist-test-slave-39l8"
server_key: "84b3b701d12e6009b0caad0ea2d34163"
server_key_iv: "cd7585e88c622dc069abdc653f68bc7d"
server_key_version: "encryptionkey@0"
I20260812 03:51:37.124840 5544 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:51:37.125444 5566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:37.125599 5544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:37.125692 5544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "2567baa83b80447d9199edae59f47274"
format_stamp: "Formatted at 2026-08-12 03:51:37 on dist-test-slave-39l8"
server_key: "84b3b701d12e6009b0caad0ea2d34163"
server_key_iv: "cd7585e88c622dc069abdc653f68bc7d"
server_key_version: "encryptionkey@0"
I20260812 03:51:37.125758 5544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:37.138031 5544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:37.138284 5544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:37.138419 5544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:37.138612 5544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:37.138921 5544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:37.138972 5544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:37.139009 5544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:37.139034 5544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:37.144255 5544 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:43721
I20260812 03:51:37.144318 5758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:43721 every 8 connection(s)
I20260812 03:51:37.144601 5544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:51:37.149292 5114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.145297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:42149 (local address 127.7.132.190:45487)
0812 03:51:37.145338 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.145340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.145542 (+ 202us) server_negotiation.cc:407] Connection header received
0812 03:51:37.146024 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.146025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.146037 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.146089 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:37.146681 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.147141 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.148042 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.148233 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.148305 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.148979 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:37.148980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:37.148980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:37.148985 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:37.149000 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:37.149003 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:37.149056 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.149188 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.149220 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:51:37.149471 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.145274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:45487 (local address 127.7.132.131:42149)
0812 03:51:37.145446 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:37.145472 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:37.145919 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:37.146166 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:37.146170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:37.146184 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:37.146588 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.146591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.147236 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.147237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.147939 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.147941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.148313 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.148315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.148452 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.148851 (+ 399us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:37.148852 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:37.148877 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:37.148895 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:37.149081 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:37.149085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:37.149095 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 03:51:37.149189 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:51:37.149279 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260812 03:51:37.149967 5759 heartbeater.cc:347] Connected to a master server at 127.7.132.190:45487
I20260812 03:51:37.150085 5759 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:37.150089 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 5544
I20260812 03:51:37.150178 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:51:37.150291 5759 heartbeater.cc:511] Master 127.7.132.190:45487 requested a full tablet report, sending...
I20260812 03:51:37.150409 7698 external_mini_cluster.cc:1428] Setting key ae999d2bfb044a239ae0872488f96b49
I20260812 03:51:37.150656 5010 ts_manager.cc:194] Registered new tserver with Master: 2567baa83b80447d9199edae59f47274 (127.7.132.131:43721)
I20260812 03:51:37.151053 5010 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.131:42149
I20260812 03:51:37.151347 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:51:37.155150 5114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.152556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.7.132.190:45487)
0812 03:51:37.152594 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.152596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.152702 (+ 106us) server_negotiation.cc:407] Connection header received
0812 03:51:37.152757 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.152758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.152769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.152804 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:37.152919 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.153314 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.153921 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.154067 (+ 146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.154113 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.154745 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:37.154746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:37.154746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:37.154750 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:37.154761 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:37.154764 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:37.154842 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.155027 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.155057 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:51:37.156723 5010 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:51:37.158102 5010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:51:37.164096 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.160103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:43721 (local address 127.0.0.1:54976)
0812 03:51:37.160399 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:37.160408 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:37.160483 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:37.160714 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:37.160717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:37.160719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:37.160785 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.160786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.161804 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.161805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.162825 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.162828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.163826 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.163827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.163907 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.163921 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:51:37.163980 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:51:37.164030 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":78,"threads_started":1}
I20260812 03:51:37.164180 5773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.160194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53572 (local address 127.7.132.129:40367)
0812 03:51:37.160648 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.160653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.160663 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:51:37.160704 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.160706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.160721 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.160806 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:37.160911 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.162114 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.163046 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.163835 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.163932 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.164015 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.164051 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.164085 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":112,"threads_started":1}
I20260812 03:51:37.164227 5769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.160100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40367 (local address 127.0.0.1:53572)
0812 03:51:37.160285 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:37.160322 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:37.160389 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:37.160810 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:37.160811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:37.160813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:37.160850 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.160851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.162234 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.162236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.162959 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.162961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.163943 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.163944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.163997 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.164001 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:51:37.164038 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 03:51:37.164075 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
I20260812 03:51:37.164529 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.160338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37549 (local address 127.0.0.1:48830)
0812 03:51:37.160450 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:37.160460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:37.160513 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:37.160997 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:37.160998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:37.161000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:37.161043 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.161045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.162206 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.162207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.163152 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.163155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.164127 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.164128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.164379 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.164383 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:51:37.164437 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:51:37.164467 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
I20260812 03:51:37.164579 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.160176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54976 (local address 127.7.132.131:43721)
0812 03:51:37.160344 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.160348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.160455 (+ 107us) server_negotiation.cc:407] Connection header received
0812 03:51:37.160588 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.160591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.160606 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.160702 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:37.160868 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.161719 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.162909 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.163740 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.163811 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.163885 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.164458 (+ 573us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.164509 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260812 03:51:37.164772 5776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.160566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.7.132.130:37549)
0812 03:51:37.160797 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.160801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.160809 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:37.160845 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.160846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.160862 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.160931 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:37.161134 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.162130 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.163234 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.163890 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.163951 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.164024 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.164647 (+ 623us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.164701 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":121,"threads_started":1}
I20260812 03:51:37.164912 5216 tablet_service.cc:1467] Processing CreateTablet for tablet a1abced8a2a0401a966dae7d890274b6 (DEFAULT_TABLE table=test-table [id=733cab882dac4aaa9e8cfcdfc11b6a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:37.165208 5216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1abced8a2a0401a966dae7d890274b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:37.167325 5646 tablet_service.cc:1467] Processing CreateTablet for tablet a1abced8a2a0401a966dae7d890274b6 (DEFAULT_TABLE table=test-table [id=733cab882dac4aaa9e8cfcdfc11b6a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:37.167496 5437 tablet_service.cc:1467] Processing CreateTablet for tablet a1abced8a2a0401a966dae7d890274b6 (DEFAULT_TABLE table=test-table [id=733cab882dac4aaa9e8cfcdfc11b6a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:37.167666 5646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1abced8a2a0401a966dae7d890274b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:37.167886 5437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1abced8a2a0401a966dae7d890274b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:37.168184 5777 tablet_bootstrap.cc:492] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf: Bootstrap starting.
I20260812 03:51:37.169005 5777 tablet_bootstrap.cc:654] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:37.169358 5777 log.cc:824] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:37.170241 5777 tablet_bootstrap.cc:492] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf: No bootstrap required, opened a new log
I20260812 03:51:37.170310 5777 ts_tablet_manager.cc:1397] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:37.170814 5779 tablet_bootstrap.cc:492] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: Bootstrap starting.
I20260812 03:51:37.171389 5779 tablet_bootstrap.cc:654] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:37.171710 5779 log.cc:824] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:37.172385 5779 tablet_bootstrap.cc:492] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: No bootstrap required, opened a new log
I20260812 03:51:37.172466 5779 ts_tablet_manager.cc:1397] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 03:51:37.172894 5777 raft_consensus.cc:348] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.173019 5777 raft_consensus.cc:374] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:37.173043 5777 raft_consensus.cc:729] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c02bc416e18348d1994c7551f39eb4bf, State: Initialized, Role: FOLLOWER
I20260812 03:51:37.173162 5777 consensus_queue.cc:260] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.173427 5777 ts_tablet_manager.cc:1428] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:51:37.173513 5323 heartbeater.cc:503] Master 127.7.132.190:45487 was elected leader, sending a full tablet report...
I20260812 03:51:37.173646 5780 tablet_bootstrap.cc:492] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: Bootstrap starting.
I20260812 03:51:37.174306 5780 tablet_bootstrap.cc:654] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:37.174645 5780 log.cc:824] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:37.174796 5779 raft_consensus.cc:348] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.174937 5779 raft_consensus.cc:374] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:37.174979 5779 raft_consensus.cc:729] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2567baa83b80447d9199edae59f47274, State: Initialized, Role: FOLLOWER
I20260812 03:51:37.175098 5779 consensus_queue.cc:260] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.175360 5779 ts_tablet_manager.cc:1428] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 03:51:37.175422 5780 tablet_bootstrap.cc:492] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: No bootstrap required, opened a new log
I20260812 03:51:37.175494 5759 heartbeater.cc:503] Master 127.7.132.190:45487 was elected leader, sending a full tablet report...
I20260812 03:51:37.175503 5780 ts_tablet_manager.cc:1397] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:37.177294 5780 raft_consensus.cc:348] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.177412 5780 raft_consensus.cc:374] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:37.177431 5780 raft_consensus.cc:729] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 421bdc5b917d4df780572a264549542d, State: Initialized, Role: FOLLOWER
I20260812 03:51:37.177544 5780 consensus_queue.cc:260] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.177745 5780 ts_tablet_manager.cc:1428] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 03:51:37.177850 5540 heartbeater.cc:503] Master 127.7.132.190:45487 was elected leader, sending a full tablet report...
I20260812 03:51:37.218595 5785 raft_consensus.cc:482] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:37.218743 5785 raft_consensus.cc:504] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.219152 5785 leader_election.cc:288] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c02bc416e18348d1994c7551f39eb4bf (127.7.132.129:40367), 421bdc5b917d4df780572a264549542d (127.7.132.130:37549)
I20260812 03:51:37.222052 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.219334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40367 (local address 127.7.132.131:33221)
0812 03:51:37.219392 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:37.219404 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:37.219449 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:37.219623 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:37.219625 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:37.219626 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:37.219696 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.219698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.220356 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.220357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.221122 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.221125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.221805 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.221806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.221871 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.221876 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:51:37.221947 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:51:37.221988 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:51:37.222052 5773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.219412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:33221 (local address 127.7.132.129:40367)
0812 03:51:37.219468 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.219470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.219480 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:51:37.219512 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.219513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.219525 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.219614 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:37.219776 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.220261 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.221228 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.221723 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.221805 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.221813 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.221956 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.221989 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:51:37.222311 5788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.219468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37549 (local address 127.7.132.131:41987)
0812 03:51:37.219667 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:37.219674 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:37.219741 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:37.219890 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:37.219891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:37.219893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:37.219989 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.219991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.220611 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.220612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.221575 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:37.221578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.222062 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.222063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:37.222136 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.222141 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:51:37.222191 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 03:51:37.222236 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260812 03:51:37.222316 5776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.219559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:41987 (local address 127.7.132.130:37549)
0812 03:51:37.219610 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.219612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.219736 (+ 124us) server_negotiation.cc:407] Connection header received
0812 03:51:37.219799 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.219800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.219808 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.219874 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:37.220059 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.220516 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.221640 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.221971 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.222049 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.222056 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.222211 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.222256 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:51:37.222378 5250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1abced8a2a0401a966dae7d890274b6" candidate_uuid: "2567baa83b80447d9199edae59f47274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c02bc416e18348d1994c7551f39eb4bf" is_pre_election: true
I20260812 03:51:37.222509 5250 raft_consensus.cc:2454] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2567baa83b80447d9199edae59f47274 in term 0.
I20260812 03:51:37.222586 5465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1abced8a2a0401a966dae7d890274b6" candidate_uuid: "2567baa83b80447d9199edae59f47274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421bdc5b917d4df780572a264549542d" is_pre_election: true
I20260812 03:51:37.222743 5571 leader_election.cc:302] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2567baa83b80447d9199edae59f47274, c02bc416e18348d1994c7551f39eb4bf; no voters:
I20260812 03:51:37.222756 5465 raft_consensus.cc:2454] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2567baa83b80447d9199edae59f47274 in term 0.
I20260812 03:51:37.222872 5785 raft_consensus.cc:2781] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:51:37.222925 5785 raft_consensus.cc:482] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:51:37.222946 5785 raft_consensus.cc:3037] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:37.223598 5785 raft_consensus.cc:504] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.223709 5785 leader_election.cc:288] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [CANDIDATE]: Term 1 election: Requested vote from peers c02bc416e18348d1994c7551f39eb4bf (127.7.132.129:40367), 421bdc5b917d4df780572a264549542d (127.7.132.130:37549)
I20260812 03:51:37.223881 5250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1abced8a2a0401a966dae7d890274b6" candidate_uuid: "2567baa83b80447d9199edae59f47274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c02bc416e18348d1994c7551f39eb4bf"
I20260812 03:51:37.223913 5465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1abced8a2a0401a966dae7d890274b6" candidate_uuid: "2567baa83b80447d9199edae59f47274" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421bdc5b917d4df780572a264549542d"
I20260812 03:51:37.223954 5250 raft_consensus.cc:3037] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:37.223996 5465 raft_consensus.cc:3037] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:37.224690 5250 raft_consensus.cc:2454] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2567baa83b80447d9199edae59f47274 in term 1.
I20260812 03:51:37.224764 5465 raft_consensus.cc:2454] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2567baa83b80447d9199edae59f47274 in term 1.
I20260812 03:51:37.224864 5571 leader_election.cc:302] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2567baa83b80447d9199edae59f47274, c02bc416e18348d1994c7551f39eb4bf; no voters:
I20260812 03:51:37.224961 5785 raft_consensus.cc:2781] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:37.225106 5785 raft_consensus.cc:686] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 1 LEADER]: Becoming Leader. State: Replica: 2567baa83b80447d9199edae59f47274, State: Running, Role: LEADER
I20260812 03:51:37.225194 5785 consensus_queue.cc:237] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } }
I20260812 03:51:37.225993 5010 catalog_manager.cc:5495] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2567baa83b80447d9199edae59f47274 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "2567baa83b80447d9199edae59f47274" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2567baa83b80447d9199edae59f47274" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 } health_report { overall_health: UNKNOWN } } }
I20260812 03:51:37.240283 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.238257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54986 (local address 127.7.132.131:43721)
0812 03:51:37.238309 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:37.238311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:37.238324 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:51:37.238349 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:37.238349 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:37.238362 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:37.238453 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:51:37.238587 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.239041 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.239681 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:37.239835 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:37.239880 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:37.240024 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:51:37.240103 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:51:37.240177 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:37.240215 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 03:51:37.240222 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:51:37.242017 5465 raft_consensus.cc:1261] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 1 FOLLOWER]: Refusing update from remote peer 2567baa83b80447d9199edae59f47274: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:37.242017 5250 raft_consensus.cc:1261] T a1abced8a2a0401a966dae7d890274b6 P c02bc416e18348d1994c7551f39eb4bf [term 1 FOLLOWER]: Refusing update from remote peer 2567baa83b80447d9199edae59f47274: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:37.242437 5789 consensus_queue.cc:1035] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "421bdc5b917d4df780572a264549542d" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:37.242563 5785 consensus_queue.cc:1035] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c02bc416e18348d1994c7551f39eb4bf" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 40367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:37.246080 5794 mvcc.cc:205] Tried to move back new op lower bound from 7317531431900663808 to 7317531431835082752. Current Snapshot: MvccSnapshot[applied={T|T < 7317531431900663808 or (T in {7317531431900663808})}]
I20260812 03:51:37.247242 5009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:37.247318 5009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:37.248100 5009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5427ad59d80e4260bd25fc640dffc594: Sending DeleteTablet for 3 replicas of tablet a1abced8a2a0401a966dae7d890274b6
I20260812 03:51:37.248427 5216 tablet_service.cc:1514] Processing DeleteTablet for tablet a1abced8a2a0401a966dae7d890274b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:37 UTC) from {username='slave'} at 127.0.0.1:53572
I20260812 03:51:37.248457 5437 tablet_service.cc:1514] Processing DeleteTablet for tablet a1abced8a2a0401a966dae7d890274b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:37 UTC) from {username='slave'} at 127.0.0.1:48830
I20260812 03:51:37.248456 5646 tablet_service.cc:1514] Processing DeleteTablet for tablet a1abced8a2a0401a966dae7d890274b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:37 UTC) from {username='slave'} at 127.0.0.1:54976
I20260812 03:51:37.248697 5817 tablet_replica.cc:330] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: stopping tablet replica
I20260812 03:51:37.248795 5817 raft_consensus.cc:2229] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:51:37.248912 5817 raft_consensus.cc:2258] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:37.248991 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 5117
I20260812 03:51:37.252159 5817 ts_tablet_manager.cc:1905] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:37.254345 5817 ts_tablet_manager.cc:1918] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:37.254441 5817 log.cc:1196] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/a1abced8a2a0401a966dae7d890274b6
I20260812 03:51:37.254748 5817 ts_tablet_manager.cc:1939] T a1abced8a2a0401a966dae7d890274b6 P 421bdc5b917d4df780572a264549542d: Deleting consensus metadata
I20260812 03:51:37.255147 4987 catalog_manager.cc:4841] TS 421bdc5b917d4df780572a264549542d (127.7.132.130:37549): tablet a1abced8a2a0401a966dae7d890274b6 (table test-table [id=733cab882dac4aaa9e8cfcdfc11b6a25]) successfully deleted
I20260812 03:51:37.248700 5812 tablet_replica.cc:330] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: stopping tablet replica
I20260812 03:51:37.256702 5812 raft_consensus.cc:2229] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:51:37.256821 5812 raft_consensus.cc:2258] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:37.257218 5812 ts_tablet_manager.cc:1905] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:37.258400 5812 ts_tablet_manager.cc:1918] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:37.258479 5812 log.cc:1196] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/a1abced8a2a0401a966dae7d890274b6
I20260812 03:51:37.258778 5812 ts_tablet_manager.cc:1939] T a1abced8a2a0401a966dae7d890274b6 P 2567baa83b80447d9199edae59f47274: Deleting consensus metadata
I20260812 03:51:37.259159 4985 catalog_manager.cc:4841] TS 2567baa83b80447d9199edae59f47274 (127.7.132.131:43721): tablet a1abced8a2a0401a966dae7d890274b6 (table test-table [id=733cab882dac4aaa9e8cfcdfc11b6a25]) successfully deleted
W20260812 03:51:37.262271 4987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:40367 (error 108)
I20260812 03:51:37.262504 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 5327
I20260812 03:51:37.265988 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:37.262653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:40367 (local address 127.0.0.1:53576)
0812 03:51:37.264663 (+ 2010us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:37.264718 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1976}
W20260812 03:51:37.266131 4987 catalog_manager.cc:4584] TS c02bc416e18348d1994c7551f39eb4bf (127.7.132.129:40367): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1abced8a2a0401a966dae7d890274b6: Network error: Client connection negotiation failed: client connection to 127.7.132.129:40367: connect: Connection refused (error 111)
I20260812 03:51:37.281909 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 5544
I20260812 03:51:37.293267 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 4961
2026-08-12T03:51:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (668 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:37 dist-test-slave-39l8 krb5kdc[5821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:37 dist-test-slave-39l8 krb5kdc[5821](info): set up 2 sockets
Aug 12 03:51:37 dist-test-slave-39l8 krb5kdc[5821](info): commencing operation
krb5kdc: starting...
W20260812 03:51:38.483071 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:38 dist-test-slave-39l8 krb5kdc[5821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:51:38.590831 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42257
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:38.667866 5844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:38.668085 5844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:38.668121 5844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:38.669198 5844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:38.669260 5844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:38.669281 5844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:38.669298 5844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:38.669328 5844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:38.670845 5844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:38.671113 5844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:38.671348 5844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:38.675841 5853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:38.675868 5854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:38.676018 5844 server_base.cc:1034] running on GCE node
W20260812 03:51:38.676034 5857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:38.676393 5844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:38.676694 5844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:38.677870 5844 hybrid_clock.cc:648] HybridClock initialized: now 1786506698677859 us; error 33 us; skew 500 ppm
Aug 12 03:51:38 dist-test-slave-39l8 krb5kdc[5821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:38.680277 5844 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:51:38.680650 5844 webserver.cc:460] Webserver started at http://127.7.132.190:36649/ using document root <none> and password file <none>
I20260812 03:51:38.680848 5844 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:38.680888 5844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:38.680971 5844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:38.681931 5844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "49b5bb341e51447e8f8675636e6bb88f"
format_stamp: "Formatted at 2026-08-12 03:51:38 on dist-test-slave-39l8"
server_key: "29e0642c007bff741967aa7595cbcb25"
server_key_iv: "e2a30b2802fd230e89246393f8f5ae6e"
server_key_version: "encryptionkey@0"
I20260812 03:51:38.682233 5844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "49b5bb341e51447e8f8675636e6bb88f"
format_stamp: "Formatted at 2026-08-12 03:51:38 on dist-test-slave-39l8"
server_key: "29e0642c007bff741967aa7595cbcb25"
server_key_iv: "e2a30b2802fd230e89246393f8f5ae6e"
server_key_version: "encryptionkey@0"
I20260812 03:51:38.683547 5844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:51:38.684208 5871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:38.684398 5844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:38.684482 5844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "49b5bb341e51447e8f8675636e6bb88f"
format_stamp: "Formatted at 2026-08-12 03:51:38 on dist-test-slave-39l8"
server_key: "29e0642c007bff741967aa7595cbcb25"
server_key_iv: "e2a30b2802fd230e89246393f8f5ae6e"
server_key_version: "encryptionkey@0"
I20260812 03:51:38.684537 5844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:38.702407 5844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:38.704512 5844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:38.704612 5844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:38.708104 5844 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42257
I20260812 03:51:38.708155 5954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42257 every 8 connection(s)
I20260812 03:51:38.708443 5844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:38.709280 5956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:38.711477 5956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f: Bootstrap starting.
I20260812 03:51:38.712096 5956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:38.712385 5956 log.cc:824] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:38.713042 5956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f: No bootstrap required, opened a new log
I20260812 03:51:38.714479 5956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b5bb341e51447e8f8675636e6bb88f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42257 } }
I20260812 03:51:38.714628 5956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:38.714708 5956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b5bb341e51447e8f8675636e6bb88f, State: Initialized, Role: FOLLOWER
I20260812 03:51:38.714900 5956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b5bb341e51447e8f8675636e6bb88f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42257 } }
I20260812 03:51:38.714998 5956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:38.715039 5956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:38.715085 5956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:38.715315 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 5844
I20260812 03:51:38.715427 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:38.715656 7698 external_mini_cluster.cc:1428] Setting key 03ca4e062a51d55e334d805fbfe1e10f
I20260812 03:51:38.715797 5956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b5bb341e51447e8f8675636e6bb88f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42257 } }
I20260812 03:51:38.715931 5956 leader_election.cc:302] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49b5bb341e51447e8f8675636e6bb88f; no voters:
I20260812 03:51:38.716080 5956 leader_election.cc:288] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:38.716100 5959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:38.716320 5959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [term 1 LEADER]: Becoming Leader. State: Replica: 49b5bb341e51447e8f8675636e6bb88f, State: Running, Role: LEADER
I20260812 03:51:38.716349 5956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:38.716449 5959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b5bb341e51447e8f8675636e6bb88f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42257 } }
I20260812 03:51:38.717120 5963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49b5bb341e51447e8f8675636e6bb88f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b5bb341e51447e8f8675636e6bb88f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42257 } } }
I20260812 03:51:38.717197 5963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:38.717229 5961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49b5bb341e51447e8f8675636e6bb88f. Latest consensus state: current_term: 1 leader_uuid: "49b5bb341e51447e8f8675636e6bb88f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49b5bb341e51447e8f8675636e6bb88f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42257 } } }
I20260812 03:51:38.717336 5961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:38.717451 5973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:38.717880 5973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 03:51:38 dist-test-slave-39l8 krb5kdc[5821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:38.726137 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:38.716104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.7.132.190:42257)
0812 03:51:38.716483 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:38.716491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:38.716520 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:51:38.716636 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:38.716642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:38.716685 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:38.716827 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:38.717598 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:38.718901 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:38.719953 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:38.720368 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:38.720454 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:38.722814 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:38.722823 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:38.722823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:38.722832 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:38.724800 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:38.725135 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:38.725138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:38.725138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:38.725234 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:38.725420 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:38.725421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:38.725421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:38.725632 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:38.725779 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:38.725861 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:51:38.725979 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":80,"threads_started":1}
I20260812 03:51:38.726688 5973 catalog_manager.cc:1347] Generated new cluster ID: 20407fec41454f3d90f2e4e2dc79d088
I20260812 03:51:38.726753 5973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:38.774755 5973 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:38.775219 5973 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:38.779671 5973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f: Generated new TSK 0
I20260812 03:51:38.779844 5973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.132.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
Entry for principal kudu/127.7.132.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab.
I20260812 03:51:38.815582 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42257
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:38.901624 5994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:38.901794 5994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:38.901815 5994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:38.902806 5994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:38.902837 5994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:38.902859 5994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:51:38.904162 5994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.129.keytab
--principal=kudu/127.7.132.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:38.904346 5994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:38.904512 5994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:38.909017 6000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:38.909173 5994 server_base.cc:1034] running on GCE node
W20260812 03:51:38.909027 6007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:38.909042 6002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:38.909435 5994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:38.909672 5994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:38.910779 5994 hybrid_clock.cc:648] HybridClock initialized: now 1786506698910763 us; error 28 us; skew 500 ppm
Aug 12 03:51:38 dist-test-slave-39l8 krb5kdc[5821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:38.913082 5994 init.cc:377] Logged in from keytab as kudu/127.7.132.129@KRBTEST.COM (short username kudu)
I20260812 03:51:38.913532 5994 webserver.cc:460] Webserver started at http://127.7.132.129:36343/ using document root <none> and password file <none>
I20260812 03:51:38.913770 5994 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:38.913839 5994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:38.913996 5994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:38.915019 5994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "7c78d60ce5b540b3bb01a29da5222ad9"
format_stamp: "Formatted at 2026-08-12 03:51:38 on dist-test-slave-39l8"
server_key: "a9b0b2c33d852596a2e4bc5347ba62cb"
server_key_iv: "07dae7613beab952b21f638bea8c558b"
server_key_version: "encryptionkey@0"
I20260812 03:51:38.915349 5994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "7c78d60ce5b540b3bb01a29da5222ad9"
format_stamp: "Formatted at 2026-08-12 03:51:38 on dist-test-slave-39l8"
server_key: "a9b0b2c33d852596a2e4bc5347ba62cb"
server_key_iv: "07dae7613beab952b21f638bea8c558b"
server_key_version: "encryptionkey@0"
I20260812 03:51:38.917124 5994 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:38.917934 6016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:38.918108 5994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:38.918208 5994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "7c78d60ce5b540b3bb01a29da5222ad9"
format_stamp: "Formatted at 2026-08-12 03:51:38 on dist-test-slave-39l8"
server_key: "a9b0b2c33d852596a2e4bc5347ba62cb"
server_key_iv: "07dae7613beab952b21f638bea8c558b"
server_key_version: "encryptionkey@0"
I20260812 03:51:38.918287 5994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:38.937506 5994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:38.939196 5994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:38.939361 5994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:38.939570 5994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:38.939908 5994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:38.939960 5994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:38.939998 5994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:38.940023 5994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:38.945649 5994 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:34095
I20260812 03:51:38.945705 6189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:34095 every 8 connection(s)
I20260812 03:51:38.945983 5994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:51:38.950899 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 5994
I20260812 03:51:38.951023 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:51:38.951269 7698 external_mini_cluster.cc:1428] Setting key 839a98e917af0fbc88ce96796d9048e1
Aug 12 03:51:38 dist-test-slave-39l8 krb5kdc[5821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506698, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.129@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:38.956025 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:38.946689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:58789 (local address 127.7.132.190:42257)
0812 03:51:38.946745 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:38.946749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:38.946965 (+ 216us) server_negotiation.cc:407] Connection header received
0812 03:51:38.947429 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:38.947430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:38.947447 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:38.947509 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:38.948437 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:38.948973 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:38.949956 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:38.950186 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:38.950267 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:38.952716 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:38.952718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:38.952719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:38.952726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:38.954176 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:38.954648 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:38.954649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:38.954649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:38.954700 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:38.955363 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:38.955364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:38.955365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:38.955489 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:38.955565 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:38.955849 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 03:51:38.955946 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:51:38.956219 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:38.946651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42257 (local address 127.7.132.129:58789)
0812 03:51:38.946860 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:38.946890 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:38.947333 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:38.947594 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:38.947598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:38.948144 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:38.948324 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:38.948329 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:38.949079 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:38.949080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:38.949869 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:38.949871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:38.950276 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:38.950277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:38.950403 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:38.950981 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:38.950994 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:38.952625 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:38.954256 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:38.954263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:38.954266 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:38.954555 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:38.954807 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:38.954808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:38.954808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:38.954969 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:38.955568 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:38.955576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:38.955709 (+ 133us) client_negotiation.cc:769] Sending connection context
0812 03:51:38.955810 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:51:38.955970 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":125,"threads_started":1}
I20260812 03:51:38.956873 6191 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42257
I20260812 03:51:38.956960 6191 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:38.957118 6191 heartbeater.cc:511] Master 127.7.132.190:42257 requested a full tablet report, sending...
I20260812 03:51:38.957683 5894 ts_manager.cc:194] Registered new tserver with Master: 7c78d60ce5b540b3bb01a29da5222ad9 (127.7.132.129:34095)
I20260812 03:51:38.958566 5894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.129@KRBTEST.COM'} at 127.7.132.129:58789
WARNING: no policy specified for kudu/127.7.132.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
Entry for principal kudu/127.7.132.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab.
I20260812 03:51:38.975711 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42257
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:39.052429 6199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:39.052604 6199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:39.052627 6199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:39.053678 6199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:39.053714 6199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:39.053761 6199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:51:39.055183 6199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.130.keytab
--principal=kudu/127.7.132.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:39.055421 6199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:39.055646 6199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:39.060308 6210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.060266 6214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.060276 6206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:39.060627 6199 server_base.cc:1034] running on GCE node
I20260812 03:51:39.060822 6199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:39.061053 6199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:39.062186 6199 hybrid_clock.cc:648] HybridClock initialized: now 1786506699062168 us; error 35 us; skew 500 ppm
Aug 12 03:51:39 dist-test-slave-39l8 krb5kdc[5821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:39.064432 6199 init.cc:377] Logged in from keytab as kudu/127.7.132.130@KRBTEST.COM (short username kudu)
I20260812 03:51:39.064853 6199 webserver.cc:460] Webserver started at http://127.7.132.130:37589/ using document root <none> and password file <none>
I20260812 03:51:39.065047 6199 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:39.065085 6199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:39.065220 6199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:39.066186 6199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "1c9383723abc4d4485fec1eddbb99f30"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "2f2ba23e6c0709df4185b5a38ed8cdd7"
server_key_iv: "6cbdc161cf380e13d3163bb2c4c2d0e4"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.066560 6199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "1c9383723abc4d4485fec1eddbb99f30"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "2f2ba23e6c0709df4185b5a38ed8cdd7"
server_key_iv: "6cbdc161cf380e13d3163bb2c4c2d0e4"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.067888 6199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:51:39.068560 6226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.068755 6199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:39.068842 6199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "1c9383723abc4d4485fec1eddbb99f30"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "2f2ba23e6c0709df4185b5a38ed8cdd7"
server_key_iv: "6cbdc161cf380e13d3163bb2c4c2d0e4"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.068909 6199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:39.080868 6199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:39.082504 6199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:39.082659 6199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:39.082867 6199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:39.083132 6199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:39.083163 6199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.083181 6199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:39.083235 6199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.088863 6199 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:33683
I20260812 03:51:39.088908 6411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:33683 every 8 connection(s)
I20260812 03:51:39.089165 6199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:51:39.090133 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 6199
I20260812 03:51:39.090293 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:51:39.090487 7698 external_mini_cluster.cc:1428] Setting key 05018814462d23f56baf9f89a4f2e7fd
Aug 12 03:51:39 dist-test-slave-39l8 krb5kdc[5821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.130@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:39.098827 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.089808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:53369 (local address 127.7.132.190:42257)
0812 03:51:39.089875 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.089877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.090087 (+ 210us) server_negotiation.cc:407] Connection header received
0812 03:51:39.090725 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.090727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.090749 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.090824 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:39.091795 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.092535 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.093507 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.093709 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.093783 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.095964 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:39.095966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:39.095966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:39.095974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:39.097556 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:39.097918 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:39.097920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:39.097920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:39.097986 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:39.098203 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:39.098204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:39.098205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:39.098350 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:39.098420 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.098631 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.098735 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 03:51:39.098949 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.089798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42257 (local address 127.7.132.130:53369)
0812 03:51:39.089960 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.089998 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.090587 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.090885 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.090888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.091522 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:39.091692 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.091696 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.092626 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.092627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.093412 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.093414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.093789 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.093790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.093913 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.094310 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:39.094320 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:39.095865 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:39.097634 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:39.097640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:39.097641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:39.097845 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:39.098057 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:39.098058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:39.098058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:39.098142 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:39.098414 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:39.098418 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:39.098509 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.098579 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.098681 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260812 03:51:39.099454 6415 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42257
I20260812 03:51:39.099545 6415 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:39.099731 6415 heartbeater.cc:511] Master 127.7.132.190:42257 requested a full tablet report, sending...
I20260812 03:51:39.100136 5894 ts_manager.cc:194] Registered new tserver with Master: 1c9383723abc4d4485fec1eddbb99f30 (127.7.132.130:33683)
WARNING: no policy specified for kudu/127.7.132.131@KRBTEST.COM; defaulting to no policy
I20260812 03:51:39.100618 5894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.130@KRBTEST.COM'} at 127.7.132.130:53369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
Entry for principal kudu/127.7.132.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab.
I20260812 03:51:39.115362 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42257
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:45633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:39.191088 6425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:39.191279 6425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:39.191303 6425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:39.192408 6425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:39.192447 6425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:39.192497 6425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:51:39.193864 6425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:45633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.131.keytab
--principal=kudu/127.7.132.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:42257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:39.194092 6425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:39.194321 6425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:39.198992 6431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:39.199059 6425 server_base.cc:1034] running on GCE node
W20260812 03:51:39.199000 6434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.199000 6432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:39.199355 6425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:39.199548 6425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:39.200680 6425 hybrid_clock.cc:648] HybridClock initialized: now 1786506699200665 us; error 27 us; skew 500 ppm
Aug 12 03:51:39 dist-test-slave-39l8 krb5kdc[5821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:39.203056 6425 init.cc:377] Logged in from keytab as kudu/127.7.132.131@KRBTEST.COM (short username kudu)
I20260812 03:51:39.203449 6425 webserver.cc:460] Webserver started at http://127.7.132.131:33827/ using document root <none> and password file <none>
I20260812 03:51:39.203677 6425 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:39.203752 6425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:39.203904 6425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:39.204856 6425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "f962157541e24fa7ba296aef5f964c79"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "4784b8f38e0de25adfed29c82d862fae"
server_key_iv: "a684c0d516a164fcee4fe19e261f092c"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.205200 6425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "f962157541e24fa7ba296aef5f964c79"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "4784b8f38e0de25adfed29c82d862fae"
server_key_iv: "a684c0d516a164fcee4fe19e261f092c"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.206457 6425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:51:39.207221 6447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.207389 6425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.207482 6425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "f962157541e24fa7ba296aef5f964c79"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "4784b8f38e0de25adfed29c82d862fae"
server_key_iv: "a684c0d516a164fcee4fe19e261f092c"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.207564 6425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:39.224526 6425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:39.226542 6425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:39.226737 6425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:39.226951 6425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:39.227224 6425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:39.227254 6425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.227304 6425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:39.227330 6425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.232630 6425 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:33877
I20260812 03:51:39.232697 6624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:33877 every 8 connection(s)
I20260812 03:51:39.232988 6425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
Aug 12 03:51:39 dist-test-slave-39l8 krb5kdc[5821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506699, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.131@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:39.240114 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 6425
I20260812 03:51:39.240264 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:51:39.240476 7698 external_mini_cluster.cc:1428] Setting key 6dae92d9a427c870f5c703e207ac0584
I20260812 03:51:39.241052 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.233630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:44551 (local address 127.7.132.190:42257)
0812 03:51:39.233698 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.233700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.233842 (+ 142us) server_negotiation.cc:407] Connection header received
0812 03:51:39.234230 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.234230 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.234239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.234292 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:39.235130 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.235568 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.236400 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.236555 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.236601 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.238518 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:39.238520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:39.238520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:39.238527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:39.239684 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:39.240048 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:39.240048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:39.240049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:39.240097 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:39.240411 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:39.240412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:39.240412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:39.240540 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:39.240613 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.240887 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.240965 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:51:39.241428 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.233591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42257 (local address 127.7.132.131:44551)
0812 03:51:39.233735 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.233769 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.234157 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.234357 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.234360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.234896 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:39.235057 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.235061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.235641 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.235641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.236332 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.236334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.236621 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.236621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.236710 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.237057 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:51:39.237064 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:39.238435 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:39.239753 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:39.239758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:39.239760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:39.239972 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:39.240181 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:51:39.240182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:51:39.240182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:51:39.240339 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:51:39.240608 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:39.240615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:39.240747 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.240859 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.241006 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260812 03:51:39.241882 6628 heartbeater.cc:347] Connected to a master server at 127.7.132.190:42257
I20260812 03:51:39.241986 6628 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:39.242149 6628 heartbeater.cc:511] Master 127.7.132.190:42257 requested a full tablet report, sending...
I20260812 03:51:39.242489 5894 ts_manager.cc:194] Registered new tserver with Master: f962157541e24fa7ba296aef5f964c79 (127.7.132.131:33877)
I20260812 03:51:39.242888 5894 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.132.131@KRBTEST.COM'} at 127.7.132.131:44551
I20260812 03:51:39.252084 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:51:39.258142 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.253349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.7.132.190:42257)
0812 03:51:39.253390 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.253393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.253487 (+ 94us) server_negotiation.cc:407] Connection header received
0812 03:51:39.253534 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.253534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.253541 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.253584 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:39.254044 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.254436 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.255104 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.255264 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.255312 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.256225 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:39.256226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:39.256226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:39.256232 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:39.257331 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:39.257603 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:39.257604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:39.257604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:39.257655 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:39.257761 (+ 106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:39.257761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:39.257761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:39.257868 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:39.257929 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.258013 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.258062 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:51:39.259455 5894 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:51:39.260984 5894 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:51:39.266640 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.263159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34095 (local address 127.0.0.1:53976)
0812 03:51:39.263355 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.263365 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.263413 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.263735 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.263735 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.263738 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:39.263772 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.263773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.264955 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.264955 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.265711 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.265714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.266379 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.266380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.266451 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.266463 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.266538 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.266582 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":51,"threads_started":1}
I20260812 03:51:39.266659 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.263397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53976 (local address 127.7.132.129:34095)
0812 03:51:39.263571 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.263575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.263583 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:39.263620 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.263622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.263636 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.263733 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:39.263910 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.264852 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.265804 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.266292 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.266377 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.266430 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.266556 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.266594 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260812 03:51:39.266803 6642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.263029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33683 (local address 127.0.0.1:36512)
0812 03:51:39.263117 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.263124 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.263180 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.263635 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.263638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.263640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:39.263700 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.263702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.264605 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.264606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.265715 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.265718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.266582 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.266583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.266650 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.266653 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.266681 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.266730 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":48,"threads_started":1}
I20260812 03:51:39.266920 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.262967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:33877 (local address 127.0.0.1:54594)
0812 03:51:39.263083 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.263119 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.263180 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.263689 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.263690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.263693 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:39.263760 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.263763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.264902 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.264903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.265870 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.265872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.266641 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.266642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.266727 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.266730 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.266770 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.266796 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":33,"thread_start_us":68,"threads_started":1}
I20260812 03:51:39.266956 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.263059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54594 (local address 127.7.132.131:33877)
0812 03:51:39.263484 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.263488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.263496 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:39.263530 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.263532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.263546 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.263617 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:39.263835 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.264803 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.265948 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.266507 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.266571 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.266614 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.266837 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.266877 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":77,"threads_started":1}
I20260812 03:51:39.267163 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.263290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36512 (local address 127.7.132.130:33683)
0812 03:51:39.263468 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.263473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.263483 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:51:39.263521 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.263523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.263538 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.263618 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:39.263780 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.264497 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.265807 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.266472 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.266962 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.267023 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.267046 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.267087 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260812 03:51:39.267539 6530 tablet_service.cc:1467] Processing CreateTablet for tablet 30c7b569a808488594d6ab018aa0dbf2 (DEFAULT_TABLE table=test-table [id=0af389cb78194811b14bc6befcce41ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:39.267675 6303 tablet_service.cc:1467] Processing CreateTablet for tablet 30c7b569a808488594d6ab018aa0dbf2 (DEFAULT_TABLE table=test-table [id=0af389cb78194811b14bc6befcce41ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:39.267833 6530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30c7b569a808488594d6ab018aa0dbf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:39.268018 6303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30c7b569a808488594d6ab018aa0dbf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:39.270309 6651 tablet_bootstrap.cc:492] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: Bootstrap starting.
I20260812 03:51:39.270658 6652 tablet_bootstrap.cc:492] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Bootstrap starting.
I20260812 03:51:39.270690 6085 tablet_service.cc:1467] Processing CreateTablet for tablet 30c7b569a808488594d6ab018aa0dbf2 (DEFAULT_TABLE table=test-table [id=0af389cb78194811b14bc6befcce41ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:39.271003 6651 tablet_bootstrap.cc:654] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:39.271049 6085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30c7b569a808488594d6ab018aa0dbf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:39.271438 6652 tablet_bootstrap.cc:654] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:39.271600 6651 log.cc:824] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:39.272094 6652 log.cc:824] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:39.272444 6651 tablet_bootstrap.cc:492] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: No bootstrap required, opened a new log
I20260812 03:51:39.272521 6651 ts_tablet_manager.cc:1397] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 03:51:39.272957 6652 tablet_bootstrap.cc:492] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: No bootstrap required, opened a new log
I20260812 03:51:39.273080 6652 ts_tablet_manager.cc:1397] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:39.274335 6657 tablet_bootstrap.cc:492] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9: Bootstrap starting.
I20260812 03:51:39.274444 6652 raft_consensus.cc:348] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.274507 6651 raft_consensus.cc:348] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.274613 6652 raft_consensus.cc:374] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:39.274633 6651 raft_consensus.cc:374] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:39.274662 6652 raft_consensus.cc:729] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9383723abc4d4485fec1eddbb99f30, State: Initialized, Role: FOLLOWER
I20260812 03:51:39.274672 6651 raft_consensus.cc:729] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f962157541e24fa7ba296aef5f964c79, State: Initialized, Role: FOLLOWER
I20260812 03:51:39.274806 6651 consensus_queue.cc:260] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.274794 6652 consensus_queue.cc:260] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.274919 6657 tablet_bootstrap.cc:654] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:39.275059 6651 ts_tablet_manager.cc:1428] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 03:51:39.275089 6652 ts_tablet_manager.cc:1428] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:39.275183 6628 heartbeater.cc:503] Master 127.7.132.190:42257 was elected leader, sending a full tablet report...
I20260812 03:51:39.275194 6415 heartbeater.cc:503] Master 127.7.132.190:42257 was elected leader, sending a full tablet report...
I20260812 03:51:39.275470 6657 log.cc:824] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:39.276242 6657 tablet_bootstrap.cc:492] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9: No bootstrap required, opened a new log
I20260812 03:51:39.276307 6657 ts_tablet_manager.cc:1397] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:39.277504 6657 raft_consensus.cc:348] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.277613 6657 raft_consensus.cc:374] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:39.277632 6657 raft_consensus.cc:729] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c78d60ce5b540b3bb01a29da5222ad9, State: Initialized, Role: FOLLOWER
I20260812 03:51:39.277700 6657 consensus_queue.cc:260] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.277890 6657 ts_tablet_manager.cc:1428] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:51:39.277959 6191 heartbeater.cc:503] Master 127.7.132.190:42257 was elected leader, sending a full tablet report...
W20260812 03:51:39.339766 6419 tablet.cc:2369] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:51:39.389837 6658 raft_consensus.cc:482] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:39.389976 6658 raft_consensus.cc:504] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.390391 6658 leader_election.cc:288] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c78d60ce5b540b3bb01a29da5222ad9 (127.7.132.129:34095), 1c9383723abc4d4485fec1eddbb99f30 (127.7.132.130:33683)
I20260812 03:51:39.393090 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.390657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:58251 (local address 127.7.132.129:34095)
0812 03:51:39.390720 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.390722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.390730 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:51:39.390759 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.390760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.390773 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.390841 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:39.391005 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.391468 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.392379 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.392747 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.392831 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.392841 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.392990 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.393030 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:51:39.393090 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.390558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34095 (local address 127.7.132.131:58251)
0812 03:51:39.390629 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.390642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.390685 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.390860 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.390861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.390863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:39.390935 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.390937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.391547 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.391548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.392284 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.392286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.392831 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.392832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.392913 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.392916 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.392983 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.393023 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 03:51:39.393414 6646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.390667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:46619 (local address 127.7.132.130:33683)
0812 03:51:39.390733 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.390736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.390920 (+ 184us) server_negotiation.cc:407] Connection header received
0812 03:51:39.391007 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.391008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.391019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.391090 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:39.391262 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.391708 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.392614 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.392986 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.393072 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.393080 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.393303 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.393343 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 03:51:39.393440 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30c7b569a808488594d6ab018aa0dbf2" candidate_uuid: "f962157541e24fa7ba296aef5f964c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" is_pre_election: true
I20260812 03:51:39.393618 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.390632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:33683 (local address 127.7.132.131:46619)
0812 03:51:39.390850 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.390858 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.390933 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.391092 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.391093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.391094 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:39.391185 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.391186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.391810 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.391811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.392522 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.392524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.393074 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.393075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.393181 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.393199 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.393511 (+ 312us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.393553 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":147,"threads_started":1}
I20260812 03:51:39.393639 6119 raft_consensus.cc:2454] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f962157541e24fa7ba296aef5f964c79 in term 0.
I20260812 03:51:39.393883 6450 leader_election.cc:302] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c78d60ce5b540b3bb01a29da5222ad9, f962157541e24fa7ba296aef5f964c79; no voters:
I20260812 03:51:39.393945 6339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30c7b569a808488594d6ab018aa0dbf2" candidate_uuid: "f962157541e24fa7ba296aef5f964c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9383723abc4d4485fec1eddbb99f30" is_pre_election: true
I20260812 03:51:39.394017 6658 raft_consensus.cc:2781] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:51:39.394083 6339 raft_consensus.cc:2454] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f962157541e24fa7ba296aef5f964c79 in term 0.
I20260812 03:51:39.394089 6658 raft_consensus.cc:482] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:51:39.394114 6658 raft_consensus.cc:3037] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:39.394886 6658 raft_consensus.cc:504] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.395022 6658 leader_election.cc:288] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [CANDIDATE]: Term 1 election: Requested vote from peers 7c78d60ce5b540b3bb01a29da5222ad9 (127.7.132.129:34095), 1c9383723abc4d4485fec1eddbb99f30 (127.7.132.130:33683)
I20260812 03:51:39.395241 6339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30c7b569a808488594d6ab018aa0dbf2" candidate_uuid: "f962157541e24fa7ba296aef5f964c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9383723abc4d4485fec1eddbb99f30"
I20260812 03:51:39.395292 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30c7b569a808488594d6ab018aa0dbf2" candidate_uuid: "f962157541e24fa7ba296aef5f964c79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c78d60ce5b540b3bb01a29da5222ad9"
I20260812 03:51:39.395357 6119 raft_consensus.cc:3037] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:39.395382 6339 raft_consensus.cc:3037] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:39.396080 6119 raft_consensus.cc:2454] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f962157541e24fa7ba296aef5f964c79 in term 1.
I20260812 03:51:39.396246 6450 leader_election.cc:302] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c78d60ce5b540b3bb01a29da5222ad9, f962157541e24fa7ba296aef5f964c79; no voters:
I20260812 03:51:39.396270 6339 raft_consensus.cc:2454] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f962157541e24fa7ba296aef5f964c79 in term 1.
I20260812 03:51:39.396346 6658 raft_consensus.cc:2781] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:39.396538 6658 raft_consensus.cc:686] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 1 LEADER]: Becoming Leader. State: Replica: f962157541e24fa7ba296aef5f964c79, State: Running, Role: LEADER
I20260812 03:51:39.396663 6658 consensus_queue.cc:237] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } }
I20260812 03:51:39.397428 5894 catalog_manager.cc:5495] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 reported cstate change: term changed from 0 to 1, leader changed from <none> to f962157541e24fa7ba296aef5f964c79 (127.7.132.131). New cstate: current_term: 1 leader_uuid: "f962157541e24fa7ba296aef5f964c79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f962157541e24fa7ba296aef5f964c79" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 } health_report { overall_health: UNKNOWN } } }
I20260812 03:51:39.409518 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.407549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54598 (local address 127.7.132.131:33877)
0812 03:51:39.407596 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.407598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.407626 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:51:39.407657 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.407657 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.407671 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.407719 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:51:39.407835 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.408227 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.408871 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.409042 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.409085 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.409204 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:51:39.409285 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:51:39.409337 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.409369 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.409383 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 03:51:39.411221 6119 raft_consensus.cc:1261] T 30c7b569a808488594d6ab018aa0dbf2 P 7c78d60ce5b540b3bb01a29da5222ad9 [term 1 FOLLOWER]: Refusing update from remote peer f962157541e24fa7ba296aef5f964c79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:39.411247 6339 raft_consensus.cc:1261] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 1 FOLLOWER]: Refusing update from remote peer f962157541e24fa7ba296aef5f964c79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:39.411586 6673 consensus_queue.cc:1035] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c9383723abc4d4485fec1eddbb99f30" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 33683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:39.411697 6658 consensus_queue.cc:1035] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c78d60ce5b540b3bb01a29da5222ad9" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 34095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:39.413964 6682 mvcc.cc:205] Tried to move back new op lower bound from 7317531440785850368 to 7317531440729333760. Current Snapshot: MvccSnapshot[applied={T|T < 7317531440785850368}]
I20260812 03:51:39.415071 6684 mvcc.cc:205] Tried to move back new op lower bound from 7317531440785850368 to 7317531440729333760. Current Snapshot: MvccSnapshot[applied={T|T < 7317531440785850368 or (T in {7317531440785850368})}]
I20260812 03:51:39.416311 5894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:39.416417 5894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:39.417161 5894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49b5bb341e51447e8f8675636e6bb88f: Sending DeleteTablet for 3 replicas of tablet 30c7b569a808488594d6ab018aa0dbf2
I20260812 03:51:39.417444 6303 tablet_service.cc:1514] Processing DeleteTablet for tablet 30c7b569a808488594d6ab018aa0dbf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:39 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:36512
I20260812 03:51:39.417560 6530 tablet_service.cc:1514] Processing DeleteTablet for tablet 30c7b569a808488594d6ab018aa0dbf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:39 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:54594
I20260812 03:51:39.417707 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 5994
I20260812 03:51:39.417753 6085 tablet_service.cc:1514] Processing DeleteTablet for tablet 30c7b569a808488594d6ab018aa0dbf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:39 UTC) from {username='kudu', principal='kudu/127.7.132.190@KRBTEST.COM'} at 127.0.0.1:53976
I20260812 03:51:39.427466 6697 tablet_replica.cc:330] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: stopping tablet replica
I20260812 03:51:39.427608 6697 raft_consensus.cc:2229] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:51:39.427748 6697 raft_consensus.cc:2258] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:39.428676 6697 ts_tablet_manager.cc:1905] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:39.430117 6697 ts_tablet_manager.cc:1918] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:39.430238 6697 log.cc:1196] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/30c7b569a808488594d6ab018aa0dbf2
I20260812 03:51:39.430490 6697 ts_tablet_manager.cc:1939] T 30c7b569a808488594d6ab018aa0dbf2 P 1c9383723abc4d4485fec1eddbb99f30: Deleting consensus metadata
I20260812 03:51:39.431020 5874 catalog_manager.cc:4841] TS 1c9383723abc4d4485fec1eddbb99f30 (127.7.132.130:33683): tablet 30c7b569a808488594d6ab018aa0dbf2 (table test-table [id=0af389cb78194811b14bc6befcce41ec]) successfully deleted
I20260812 03:51:39.431474 6699 tablet_replica.cc:330] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: stopping tablet replica
I20260812 03:51:39.431605 6699 raft_consensus.cc:2229] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:51:39.431711 6699 raft_consensus.cc:2258] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:39.432085 6699 ts_tablet_manager.cc:1905] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:39.433256 6699 ts_tablet_manager.cc:1918] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:39.433346 6699 log.cc:1196] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/30c7b569a808488594d6ab018aa0dbf2
I20260812 03:51:39.433593 6699 ts_tablet_manager.cc:1939] T 30c7b569a808488594d6ab018aa0dbf2 P f962157541e24fa7ba296aef5f964c79: Deleting consensus metadata
I20260812 03:51:39.433969 5875 catalog_manager.cc:4841] TS f962157541e24fa7ba296aef5f964c79 (127.7.132.131:33877): tablet 30c7b569a808488594d6ab018aa0dbf2 (table test-table [id=0af389cb78194811b14bc6befcce41ec]) successfully deleted
W20260812 03:51:39.434109 5875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:34095 (error 108)
I20260812 03:51:39.434494 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 6199
I20260812 03:51:39.434567 6640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.434426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:34095 (local address 127.0.0.1:53984)
0812 03:51:39.434480 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.434515 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:34095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 03:51:39.436694 5875 catalog_manager.cc:4584] TS 7c78d60ce5b540b3bb01a29da5222ad9 (127.7.132.129:34095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30c7b569a808488594d6ab018aa0dbf2: Network error: Client connection negotiation failed: client connection to 127.7.132.129:34095: connect: Connection refused (error 111)
I20260812 03:51:39.441918 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 6425
I20260812 03:51:39.454053 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 5844
2026-08-12T03:51:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2160 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T03:51:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:39Z Disabled control of system clock
I20260812 03:51:39.485643 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39619
--webserver_interface=127.7.132.190
--webserver_port=0
--builtin_ntp_servers=127.7.132.148:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:39619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:39.560199 6705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:39.560401 6705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:39.560422 6705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:39.561470 6705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:39.561503 6705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:39.561517 6705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:39.561528 6705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:39.561540 6705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:39.562736 6705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:39619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:39619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:39.562891 6705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:39.563097 6705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:39.567423 6718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.567461 6723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.567464 6719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:39.567713 6705 server_base.cc:1034] running on GCE node
I20260812 03:51:39.567960 6705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:39.568251 6705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:39.569424 6705 hybrid_clock.cc:648] HybridClock initialized: now 1786506699569412 us; error 24 us; skew 500 ppm
I20260812 03:51:39.570619 6705 webserver.cc:460] Webserver started at http://127.7.132.190:36939/ using document root <none> and password file <none>
I20260812 03:51:39.571036 6705 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:39.571090 6705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:39.571281 6705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:39.572311 6705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "6bc5c5220a054ec7a62c09e864e4dc4f"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "1f6e96941642432608d1bca1c86a3905"
server_key_iv: "4c5572610c57d24318443dbf583444e6"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.572667 6705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "6bc5c5220a054ec7a62c09e864e4dc4f"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "1f6e96941642432608d1bca1c86a3905"
server_key_iv: "4c5572610c57d24318443dbf583444e6"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.573920 6705 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:51:39.574615 6729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.574801 6705 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:39.574901 6705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "6bc5c5220a054ec7a62c09e864e4dc4f"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "1f6e96941642432608d1bca1c86a3905"
server_key_iv: "4c5572610c57d24318443dbf583444e6"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.574971 6705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:39.592132 6705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:39.592414 6705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:39.592566 6705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:39.596040 6705 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:39619
I20260812 03:51:39.596097 6808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:39619 every 8 connection(s)
I20260812 03:51:39.596392 6705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:39.597245 6810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:39.599255 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f: Bootstrap starting.
I20260812 03:51:39.599635 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 6705
I20260812 03:51:39.599732 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:39.599916 6810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:39.599953 7698 external_mini_cluster.cc:1428] Setting key 3544bcbe3c68690c22fb968be240132f
I20260812 03:51:39.600260 6810 log.cc:824] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:39.601243 6810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f: No bootstrap required, opened a new log
I20260812 03:51:39.603163 6810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc5c5220a054ec7a62c09e864e4dc4f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39619 } }
I20260812 03:51:39.603310 6810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:39.603371 6810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bc5c5220a054ec7a62c09e864e4dc4f, State: Initialized, Role: FOLLOWER
I20260812 03:51:39.603502 6810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc5c5220a054ec7a62c09e864e4dc4f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39619 } }
I20260812 03:51:39.603607 6810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:39.603652 6810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:39.603698 6810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:39.604336 6810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc5c5220a054ec7a62c09e864e4dc4f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39619 } }
I20260812 03:51:39.604457 6810 leader_election.cc:302] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bc5c5220a054ec7a62c09e864e4dc4f; no voters:
I20260812 03:51:39.604626 6810 leader_election.cc:288] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:39.604703 6819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:39.604878 6819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [term 1 LEADER]: Becoming Leader. State: Replica: 6bc5c5220a054ec7a62c09e864e4dc4f, State: Running, Role: LEADER
I20260812 03:51:39.604964 6810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:39.605082 6819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc5c5220a054ec7a62c09e864e4dc4f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39619 } }
I20260812 03:51:39.605458 6819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bc5c5220a054ec7a62c09e864e4dc4f. Latest consensus state: current_term: 1 leader_uuid: "6bc5c5220a054ec7a62c09e864e4dc4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc5c5220a054ec7a62c09e864e4dc4f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39619 } } }
I20260812 03:51:39.605636 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.600415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.7.132.190:39619)
0812 03:51:39.600616 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.600622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.600646 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:51:39.601017 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.601022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.601073 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.601180 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:39.601455 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.602279 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.603385 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.603917 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.604004 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.604703 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:39.604706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:39.604707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:39.604947 (+ 240us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:39.604992 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:39.605007 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:39.605241 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.605276 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.605371 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":138,"thread_start_us":93,"threads_started":1}
I20260812 03:51:39.605669 6819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:39.605643 6822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bc5c5220a054ec7a62c09e864e4dc4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc5c5220a054ec7a62c09e864e4dc4f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 39619 } } }
I20260812 03:51:39.605927 6822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:39.606309 6831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:39.606779 6831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:39.608160 6831 catalog_manager.cc:1347] Generated new cluster ID: 95f885750d1d4a56b57e48cbce172ba7
I20260812 03:51:39.608208 6831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:39.642194 6831 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:39.642879 6831 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:39.647667 6831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f: Generated new TSK 0
I20260812 03:51:39.647840 6831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:39.652695 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.129:0
--local_ip_for_outbound_sockets=127.7.132.129
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39619
--builtin_ntp_servers=127.7.132.148:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:39.735091 6855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:39.735333 6855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:39.735356 6855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:39.736516 6855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:39.736558 6855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:39.736594 6855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.129
I20260812 03:51:39.737836 6855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.132.129
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.129
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:39.738061 6855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:39.738336 6855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:39.743180 6861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.743192 6862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.743192 6864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:39.743446 6855 server_base.cc:1034] running on GCE node
I20260812 03:51:39.743644 6855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:39.743873 6855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:39.745054 6855 hybrid_clock.cc:648] HybridClock initialized: now 1786506699745018 us; error 32 us; skew 500 ppm
I20260812 03:51:39.746332 6855 webserver.cc:460] Webserver started at http://127.7.132.129:43395/ using document root <none> and password file <none>
I20260812 03:51:39.746569 6855 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:39.746640 6855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:39.746800 6855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:39.747805 6855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/data/instance:
uuid: "c726bef1ed38498d8b064234865f479a"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "b86eee8a1dcd2ecfd6d62b0a3d4d2b4d"
server_key_iv: "c272e60e2c4bb23a7e26b2610ba60bd9"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.748174 6855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance:
uuid: "c726bef1ed38498d8b064234865f479a"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "b86eee8a1dcd2ecfd6d62b0a3d4d2b4d"
server_key_iv: "c272e60e2c4bb23a7e26b2610ba60bd9"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.749440 6855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:51:39.750200 6871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.750387 6855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:39.750496 6855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/wal
uuid: "c726bef1ed38498d8b064234865f479a"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "b86eee8a1dcd2ecfd6d62b0a3d4d2b4d"
server_key_iv: "c272e60e2c4bb23a7e26b2610ba60bd9"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.750571 6855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:39.784307 6855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:39.784662 6855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:39.784861 6855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:39.785115 6855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:39.785472 6855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:39.785529 6855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.785575 6855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:39.785601 6855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.791296 6855 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.129:45249
I20260812 03:51:39.791359 7055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.129:45249 every 8 connection(s)
I20260812 03:51:39.791671 6855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/data/info.pb
I20260812 03:51:39.796200 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.792365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.129:59277 (local address 127.7.132.190:39619)
0812 03:51:39.792442 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.792444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.792663 (+ 219us) server_negotiation.cc:407] Connection header received
0812 03:51:39.793121 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.793122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.793132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.793183 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:39.793758 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.794175 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.795045 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.795215 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.795268 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.796018 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:39.796019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:39.796020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:39.796023 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:39.796033 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:39.796036 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:39.796068 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.796130 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.796146 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:51:39.796414 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.792338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39619 (local address 127.7.132.129:59277)
0812 03:51:39.792544 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.792578 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.793042 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.793235 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.793238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.793247 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:39.793700 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.793703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.794246 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.794247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.794962 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.794964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.795288 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.795289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.795384 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.795904 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:39.795905 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:39.795932 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:39.795953 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:39.796070 (+ 117us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:39.796072 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:39.796074 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.796115 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.796190 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260812 03:51:39.796919 7056 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39619
I20260812 03:51:39.796993 7056 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:39.797187 7056 heartbeater.cc:511] Master 127.7.132.190:39619 requested a full tablet report, sending...
I20260812 03:51:39.797681 6754 ts_manager.cc:194] Registered new tserver with Master: c726bef1ed38498d8b064234865f479a (127.7.132.129:45249)
I20260812 03:51:39.798214 6754 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.129:59277
I20260812 03:51:39.798688 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 6855
I20260812 03:51:39.798843 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-0/wal/instance
I20260812 03:51:39.799081 7698 external_mini_cluster.cc:1428] Setting key 9244c4a037e704e5fcfc012017670167
I20260812 03:51:39.799889 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.130:0
--local_ip_for_outbound_sockets=127.7.132.130
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39619
--builtin_ntp_servers=127.7.132.148:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:51:39.876447 7063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:39.876653 7063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:39.876688 7063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:39.877877 7063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:39.877920 7063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:39.877959 7063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.130
I20260812 03:51:39.879261 7063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.132.130
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.130
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:39.879501 7063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:39.879794 7063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:39.884456 7072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.884466 7077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:39.884488 7073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:39.884784 7063 server_base.cc:1034] running on GCE node
I20260812 03:51:39.884995 7063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:39.885217 7063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:39.886372 7063 hybrid_clock.cc:648] HybridClock initialized: now 1786506699886366 us; error 34 us; skew 500 ppm
I20260812 03:51:39.887542 7063 webserver.cc:460] Webserver started at http://127.7.132.130:46013/ using document root <none> and password file <none>
I20260812 03:51:39.887764 7063 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:39.887828 7063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:39.887950 7063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:39.888935 7063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/data/instance:
uuid: "94fd6cb095974990aaea469de773db8b"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "8f161d8b559b956221f37ced395a1a3e"
server_key_iv: "9905adbaf21867d5a3ca728c597a275d"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.889290 7063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance:
uuid: "94fd6cb095974990aaea469de773db8b"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "8f161d8b559b956221f37ced395a1a3e"
server_key_iv: "9905adbaf21867d5a3ca728c597a275d"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.890525 7063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:51:39.891300 7087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.891477 7063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.891574 7063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/wal
uuid: "94fd6cb095974990aaea469de773db8b"
format_stamp: "Formatted at 2026-08-12 03:51:39 on dist-test-slave-39l8"
server_key: "8f161d8b559b956221f37ced395a1a3e"
server_key_iv: "9905adbaf21867d5a3ca728c597a275d"
server_key_version: "encryptionkey@0"
I20260812 03:51:39.891660 7063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:39.904740 7063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:39.904973 7063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:39.905098 7063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:39.905333 7063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:39.905640 7063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:39.905691 7063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.905735 7063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:39.905758 7063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:39.911183 7063 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.130:37301
I20260812 03:51:39.911237 7272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.130:37301 every 8 connection(s)
I20260812 03:51:39.911552 7063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/data/info.pb
I20260812 03:51:39.915128 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 7063
I20260812 03:51:39.915251 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/wal/instance
I20260812 03:51:39.915472 7698 external_mini_cluster.cc:1428] Setting key a53c37a17fb1bf480bd956c713703014
I20260812 03:51:39.916510 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.912236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.130:59925 (local address 127.7.132.190:39619)
0812 03:51:39.912298 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:39.912300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:39.912516 (+ 216us) server_negotiation.cc:407] Connection header received
0812 03:51:39.912929 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:39.912930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:39.912940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:39.912992 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:39.913556 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.914070 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.914992 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.915296 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.915367 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.916188 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:39.916189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:39.916190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:39.916194 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:39.916209 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:39.916213 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:39.916269 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:39.916370 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 03:51:39.916396 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 03:51:39.916535 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.132.131:0
--local_ip_for_outbound_sockets=127.7.132.131
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39619
--builtin_ntp_servers=127.7.132.148:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:51:39.916584 7276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:39.912200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39619 (local address 127.7.132.130:59925)
0812 03:51:39.912376 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:39.912437 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:39.912853 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:39.913035 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:39.913038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:39.913045 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:39.913492 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.913495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.914143 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.914144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.914887 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:39.914889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:39.915383 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:39.915384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:39.915486 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:39.916039 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:39.916040 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:39.916074 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:39.916098 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:39.916282 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:39.916288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:39.916290 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 03:51:39.916361 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:51:39.916428 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260812 03:51:39.917008 7274 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39619
I20260812 03:51:39.917105 7274 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:39.917259 7274 heartbeater.cc:511] Master 127.7.132.190:39619 requested a full tablet report, sending...
I20260812 03:51:39.917593 6754 ts_manager.cc:194] Registered new tserver with Master: 94fd6cb095974990aaea469de773db8b (127.7.132.130:37301)
I20260812 03:51:39.918046 6754 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.130:59925
W20260812 03:51:39.996042 7279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:39.996219 7279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:39.996245 7279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:39.997267 7279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:39.997303 7279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:39.997325 7279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.132.131
I20260812 03:51:39.998553 7279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.132.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.132.131
--webserver_port=0
--tserver_master_addrs=127.7.132.190:39619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.132.131
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:39.998798 7279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:39.999066 7279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:40.003796 7292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:40.003757 7290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:40.003757 7296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:40.003813 7279 server_base.cc:1034] running on GCE node
I20260812 03:51:40.004184 7279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:40.004415 7279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:40.005571 7279 hybrid_clock.cc:648] HybridClock initialized: now 1786506700005557 us; error 44 us; skew 500 ppm
I20260812 03:51:40.006836 7279 webserver.cc:460] Webserver started at http://127.7.132.131:46639/ using document root <none> and password file <none>
I20260812 03:51:40.007061 7279 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:40.007122 7279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:40.007259 7279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:40.008205 7279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/data/instance:
uuid: "41f69e8d2e0e41d98fa253cea2b644fd"
format_stamp: "Formatted at 2026-08-12 03:51:40 on dist-test-slave-39l8"
server_key: "28fb9a73a05bfbab8fe331dd27d861ac"
server_key_iv: "ddd526edbc0914e1a385b92dbc2d373b"
server_key_version: "encryptionkey@0"
I20260812 03:51:40.008543 7279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance:
uuid: "41f69e8d2e0e41d98fa253cea2b644fd"
format_stamp: "Formatted at 2026-08-12 03:51:40 on dist-test-slave-39l8"
server_key: "28fb9a73a05bfbab8fe331dd27d861ac"
server_key_iv: "ddd526edbc0914e1a385b92dbc2d373b"
server_key_version: "encryptionkey@0"
I20260812 03:51:40.009727 7279 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:51:40.010485 7307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:40.010668 7279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:40.010785 7279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/wal
uuid: "41f69e8d2e0e41d98fa253cea2b644fd"
format_stamp: "Formatted at 2026-08-12 03:51:40 on dist-test-slave-39l8"
server_key: "28fb9a73a05bfbab8fe331dd27d861ac"
server_key_iv: "ddd526edbc0914e1a385b92dbc2d373b"
server_key_version: "encryptionkey@0"
I20260812 03:51:40.010874 7279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:40.032724 7279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:40.033007 7279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:40.033156 7279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:40.033368 7279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:51:40.033696 7279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:51:40.033756 7279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:40.033787 7279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:51:40.033812 7279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:40.039188 7279 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.131:43911
I20260812 03:51:40.039309 7476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.131:43911 every 8 connection(s)
I20260812 03:51:40.039604 7279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/data/info.pb
I20260812 03:51:40.042124 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 7279
I20260812 03:51:40.042249 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/wal/instance
I20260812 03:51:40.042496 7698 external_mini_cluster.cc:1428] Setting key 02d1b0598a71d181a5c91bf70df24b86
I20260812 03:51:40.045060 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.040380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:34789 (local address 127.7.132.190:39619)
0812 03:51:40.040465 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:40.040467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:40.040715 (+ 248us) server_negotiation.cc:407] Connection header received
0812 03:51:40.041256 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:40.041258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:40.041271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:40.041343 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:40.041985 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.042755 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.043734 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.043943 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.044015 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.044796 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:40.044797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:40.044798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:40.044802 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:40.044816 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:40.044820 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:40.044870 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:40.044964 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 03:51:40.044987 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:51:40.045274 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.040340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:39619 (local address 127.7.132.131:34789)
0812 03:51:40.040585 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:40.040619 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:40.041147 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:40.041426 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:40.041429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:40.041437 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:51:40.041891 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.041895 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.042854 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.042854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.043618 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.043620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.044036 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.044037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.044181 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.044640 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:51:40.044641 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:51:40.044676 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:51:40.044698 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:51:40.044874 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:51:40.044878 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:51:40.044881 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:40.044953 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:51:40.045044 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260812 03:51:40.045750 7477 heartbeater.cc:347] Connected to a master server at 127.7.132.190:39619
I20260812 03:51:40.045837 7477 heartbeater.cc:464] Registering TS with master...
I20260812 03:51:40.046017 7477 heartbeater.cc:511] Master 127.7.132.190:39619 requested a full tablet report, sending...
I20260812 03:51:40.046350 6754 ts_manager.cc:194] Registered new tserver with Master: 41f69e8d2e0e41d98fa253cea2b644fd (127.7.132.131:43911)
I20260812 03:51:40.046685 6754 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.132.131:34789
I20260812 03:51:40.053944 7698 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:51:40.057569 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.055338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.7.132.190:39619)
0812 03:51:40.055395 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:40.055397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:40.055483 (+ 86us) server_negotiation.cc:407] Connection header received
0812 03:51:40.055529 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:40.055530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:40.055537 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:40.055580 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:40.055686 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.056057 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.056686 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.056844 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.056889 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.057393 (+ 504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:40.057394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:40.057394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:40.057398 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:40.057408 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:40.057411 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:40.057447 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:40.057475 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 03:51:40.057490 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:51:40.058943 6754 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:51:40.060483 6754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:51:40.066432 7503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.062680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45249 (local address 127.0.0.1:48336)
0812 03:51:40.063304 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:40.063313 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:40.063357 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:40.063631 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:40.063632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:40.063633 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:40.063671 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.063672 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.064779 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.064780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.065478 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.065480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.066102 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.066103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.066200 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.066214 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 03:51:40.066311 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:51:40.066362 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":100,"threads_started":1}
I20260812 03:51:40.066449 7501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.063071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.7.132.129:45249)
0812 03:51:40.063251 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:40.063255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:40.063420 (+ 165us) server_negotiation.cc:407] Connection header received
0812 03:51:40.063469 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:40.063471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:40.063501 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:40.063566 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:40.063745 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.064699 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.065557 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.066018 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.066100 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.066205 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:40.066326 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 03:51:40.066365 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260812 03:51:40.066627 7498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.062643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.131:43911 (local address 127.0.0.1:53760)
0812 03:51:40.062850 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:40.062885 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:40.062957 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:40.063216 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:40.063220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:40.063222 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:40.063276 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.063279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.064630 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.064631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.065521 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.065522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.066427 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.066428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.066481 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.066484 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:40.066529 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 03:51:40.066561 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
I20260812 03:51:40.066756 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.062658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37301 (local address 127.0.0.1:33706)
0812 03:51:40.063014 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:40.063024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:40.063076 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:40.063302 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:40.063303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:40.063305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:40.063349 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.063350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.064303 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.064304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.065334 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.065336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.066259 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.066280 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.066341 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.066345 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:51:40.066392 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 03:51:40.066434 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":589,"threads_started":1}
I20260812 03:51:40.066937 7499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.062769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53760 (local address 127.7.132.131:43911)
0812 03:51:40.062919 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:40.062923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:40.062938 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:51:40.063069 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:40.063071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:40.063085 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:40.063151 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:40.063354 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.064545 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.065594 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.066240 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.066320 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.066396 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:40.066818 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 03:51:40.066866 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260812 03:51:40.067006 7500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.062856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.7.132.130:37301)
0812 03:51:40.063045 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:40.063049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:40.063061 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:51:40.063141 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:40.063143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:40.063157 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:40.063231 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:40.063426 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.064218 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.065418 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.066054 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.066116 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.066213 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:40.066477 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:51:40.066913 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260812 03:51:40.067390 6940 tablet_service.cc:1467] Processing CreateTablet for tablet 365415276f014f1ca93d9a3b9809b47a (DEFAULT_TABLE table=test-table [id=62d8d0ef9d20416f8b91110394344af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:40.067503 7371 tablet_service.cc:1467] Processing CreateTablet for tablet 365415276f014f1ca93d9a3b9809b47a (DEFAULT_TABLE table=test-table [id=62d8d0ef9d20416f8b91110394344af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:40.067643 7163 tablet_service.cc:1467] Processing CreateTablet for tablet 365415276f014f1ca93d9a3b9809b47a (DEFAULT_TABLE table=test-table [id=62d8d0ef9d20416f8b91110394344af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:51:40.067788 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365415276f014f1ca93d9a3b9809b47a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:40.067826 7371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365415276f014f1ca93d9a3b9809b47a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:40.067927 7163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 365415276f014f1ca93d9a3b9809b47a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:40.070284 7505 tablet_bootstrap.cc:492] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: Bootstrap starting.
I20260812 03:51:40.071343 7507 tablet_bootstrap.cc:492] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: Bootstrap starting.
I20260812 03:51:40.071496 7505 tablet_bootstrap.cc:654] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:40.071830 7505 log.cc:824] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:40.072010 7507 tablet_bootstrap.cc:654] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:40.072110 7508 tablet_bootstrap.cc:492] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a: Bootstrap starting.
I20260812 03:51:40.072355 7507 log.cc:824] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:40.072670 7505 tablet_bootstrap.cc:492] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: No bootstrap required, opened a new log
I20260812 03:51:40.072748 7505 ts_tablet_manager.cc:1397] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:51:40.072782 7508 tablet_bootstrap.cc:654] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:40.073089 7508 log.cc:824] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:40.073235 7507 tablet_bootstrap.cc:492] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: No bootstrap required, opened a new log
I20260812 03:51:40.073307 7507 ts_tablet_manager.cc:1397] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:51:40.074568 7508 tablet_bootstrap.cc:492] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a: No bootstrap required, opened a new log
I20260812 03:51:40.074635 7508 ts_tablet_manager.cc:1397] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:51:40.075421 7507 raft_consensus.cc:348] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.075534 7507 raft_consensus.cc:374] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:40.075398 7505 raft_consensus.cc:348] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.075553 7507 raft_consensus.cc:729] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94fd6cb095974990aaea469de773db8b, State: Initialized, Role: FOLLOWER
I20260812 03:51:40.075567 7505 raft_consensus.cc:374] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:40.075598 7505 raft_consensus.cc:729] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f69e8d2e0e41d98fa253cea2b644fd, State: Initialized, Role: FOLLOWER
I20260812 03:51:40.075662 7507 consensus_queue.cc:260] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.075704 7505 consensus_queue.cc:260] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.075934 7507 ts_tablet_manager.cc:1428] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:51:40.075953 7274 heartbeater.cc:503] Master 127.7.132.190:39619 was elected leader, sending a full tablet report...
I20260812 03:51:40.075954 7505 ts_tablet_manager.cc:1428] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 03:51:40.076059 7477 heartbeater.cc:503] Master 127.7.132.190:39619 was elected leader, sending a full tablet report...
I20260812 03:51:40.077168 7508 raft_consensus.cc:348] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.077275 7508 raft_consensus.cc:374] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:40.077291 7508 raft_consensus.cc:729] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c726bef1ed38498d8b064234865f479a, State: Initialized, Role: FOLLOWER
I20260812 03:51:40.077402 7508 consensus_queue.cc:260] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.077632 7508 ts_tablet_manager.cc:1428] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 03:51:40.077647 7056 heartbeater.cc:503] Master 127.7.132.190:39619 was elected leader, sending a full tablet report...
I20260812 03:51:40.110992 7519 raft_consensus.cc:482] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:51:40.111104 7519 raft_consensus.cc:504] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.111428 7519 leader_election.cc:288] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c726bef1ed38498d8b064234865f479a (127.7.132.129:45249), 94fd6cb095974990aaea469de773db8b (127.7.132.130:37301)
I20260812 03:51:40.114843 7501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.111673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:51401 (local address 127.7.132.129:45249)
0812 03:51:40.111718 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:40.111720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:40.111898 (+ 178us) server_negotiation.cc:407] Connection header received
0812 03:51:40.111938 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:40.111939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:40.111947 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:40.112007 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:40.112153 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.112802 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.114038 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.114534 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.114598 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.114605 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:40.114759 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:51:40.114791 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:51:40.114843 7500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.111650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.132.131:60467 (local address 127.7.132.130:37301)
0812 03:51:40.111690 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:40.111692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:40.111702 (+ 10us) server_negotiation.cc:407] Connection header received
0812 03:51:40.111762 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:40.111763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:40.111772 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:40.111838 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:51:40.111996 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.112733 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.114043 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.114534 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.114599 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.114605 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:40.114759 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:51:40.114789 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:51:40.114884 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.111575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.130:37301 (local address 127.7.132.131:60467)
0812 03:51:40.111634 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:40.111645 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:40.111704 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:40.111855 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:40.111856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:40.111857 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:40.111923 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.111925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.112805 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.112805 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.113954 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.113957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.114605 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.114606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.114673 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.114676 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:40.114756 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:51:40.114803 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":34}
I20260812 03:51:40.115072 7527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.111575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45249 (local address 127.7.132.131:51401)
0812 03:51:40.111828 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:40.111837 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 03:51:40.111884 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:51:40.112021 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:51:40.112022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:51:40.112033 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:51:40.112090 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.112091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.112871 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.112871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.113954 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:51:40.113957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.114606 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.114606 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:51:40.114676 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.114679 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:51:40.114755 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:51:40.114802 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":105,"thread_start_us":100,"threads_started":1}
I20260812 03:51:40.115253 7194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365415276f014f1ca93d9a3b9809b47a" candidate_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94fd6cb095974990aaea469de773db8b" is_pre_election: true
I20260812 03:51:40.115401 7194 raft_consensus.cc:2454] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f69e8d2e0e41d98fa253cea2b644fd in term 0.
I20260812 03:51:40.115392 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365415276f014f1ca93d9a3b9809b47a" candidate_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c726bef1ed38498d8b064234865f479a" is_pre_election: true
I20260812 03:51:40.115500 6969 raft_consensus.cc:2454] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41f69e8d2e0e41d98fa253cea2b644fd in term 0.
I20260812 03:51:40.115609 7308 leader_election.cc:302] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f69e8d2e0e41d98fa253cea2b644fd, 94fd6cb095974990aaea469de773db8b; no voters:
I20260812 03:51:40.115720 7519 raft_consensus.cc:2781] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:51:40.115777 7519 raft_consensus.cc:482] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:51:40.115797 7519 raft_consensus.cc:3037] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:40.116361 7519 raft_consensus.cc:504] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.116487 7519 leader_election.cc:288] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [CANDIDATE]: Term 1 election: Requested vote from peers c726bef1ed38498d8b064234865f479a (127.7.132.129:45249), 94fd6cb095974990aaea469de773db8b (127.7.132.130:37301)
I20260812 03:51:40.116598 6969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365415276f014f1ca93d9a3b9809b47a" candidate_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c726bef1ed38498d8b064234865f479a"
I20260812 03:51:40.116627 7194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "365415276f014f1ca93d9a3b9809b47a" candidate_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94fd6cb095974990aaea469de773db8b"
I20260812 03:51:40.116683 7194 raft_consensus.cc:3037] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:40.116668 6969 raft_consensus.cc:3037] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:40.117446 6969 raft_consensus.cc:2454] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f69e8d2e0e41d98fa253cea2b644fd in term 1.
I20260812 03:51:40.117446 7194 raft_consensus.cc:2454] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41f69e8d2e0e41d98fa253cea2b644fd in term 1.
I20260812 03:51:40.117605 7310 leader_election.cc:302] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41f69e8d2e0e41d98fa253cea2b644fd, c726bef1ed38498d8b064234865f479a; no voters:
I20260812 03:51:40.117722 7519 raft_consensus.cc:2781] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:40.117825 7519 raft_consensus.cc:686] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 1 LEADER]: Becoming Leader. State: Replica: 41f69e8d2e0e41d98fa253cea2b644fd, State: Running, Role: LEADER
I20260812 03:51:40.117882 7519 consensus_queue.cc:237] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } }
I20260812 03:51:40.118597 6753 catalog_manager.cc:5495] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 41f69e8d2e0e41d98fa253cea2b644fd (127.7.132.131). New cstate: current_term: 1 leader_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f69e8d2e0e41d98fa253cea2b644fd" member_type: VOTER last_known_addr { host: "127.7.132.131" port: 43911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 } health_report { overall_health: UNKNOWN } } }
I20260812 03:51:40.122890 7499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.121073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53770 (local address 127.7.132.131:43911)
0812 03:51:40.121120 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:40.121122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:40.121129 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:51:40.121158 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:40.121159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:40.121180 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:40.121231 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:51:40.121313 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.121689 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.122292 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:40.122440 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:40.122483 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:40.122589 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:51:40.122659 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:51:40.122757 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:40.122772 (+ 15us) server_negotiation.cc:299] Negotiation successful
0812 03:51:40.122786 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:51:40.124253 6969 raft_consensus.cc:1261] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [term 1 FOLLOWER]: Refusing update from remote peer 41f69e8d2e0e41d98fa253cea2b644fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:40.124312 7194 raft_consensus.cc:1261] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 1 FOLLOWER]: Refusing update from remote peer 41f69e8d2e0e41d98fa253cea2b644fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:51:40.124569 7529 consensus_queue.cc:1035] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c726bef1ed38498d8b064234865f479a" member_type: VOTER last_known_addr { host: "127.7.132.129" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:40.124666 7519 consensus_queue.cc:1035] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "94fd6cb095974990aaea469de773db8b" member_type: VOTER last_known_addr { host: "127.7.132.130" port: 37301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:51:40.127000 7533 mvcc.cc:205] Tried to move back new op lower bound from 7317531443706781696 to 7317531443683450880. Current Snapshot: MvccSnapshot[applied={T|T < 7317531443706781696}]
I20260812 03:51:40.127452 7535 mvcc.cc:205] Tried to move back new op lower bound from 7317531443706781696 to 7317531443683450880. Current Snapshot: MvccSnapshot[applied={T|T < 7317531443706781696 or (T in {7317531443706781696})}]
I20260812 03:51:40.128983 6753 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:40.129066 6753 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:51:40.129688 6753 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bc5c5220a054ec7a62c09e864e4dc4f: Sending DeleteTablet for 3 replicas of tablet 365415276f014f1ca93d9a3b9809b47a
I20260812 03:51:40.129985 7371 tablet_service.cc:1514] Processing DeleteTablet for tablet 365415276f014f1ca93d9a3b9809b47a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:40 UTC) from {username='slave'} at 127.0.0.1:53760
I20260812 03:51:40.130030 6940 tablet_service.cc:1514] Processing DeleteTablet for tablet 365415276f014f1ca93d9a3b9809b47a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:40 UTC) from {username='slave'} at 127.0.0.1:48336
I20260812 03:51:40.130121 7163 tablet_service.cc:1514] Processing DeleteTablet for tablet 365415276f014f1ca93d9a3b9809b47a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:51:40 UTC) from {username='slave'} at 127.0.0.1:33706
I20260812 03:51:40.130225 7545 tablet_replica.cc:330] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a: stopping tablet replica
I20260812 03:51:40.130322 7545 raft_consensus.cc:2229] T 365415276f014f1ca93d9a3b9809b47a P c726bef1ed38498d8b064234865f479a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:51:40.130398 7546 tablet_replica.cc:330] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: stopping tablet replica
I20260812 03:51:40.130494 7546 raft_consensus.cc:2229] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:51:40.130584 7546 raft_consensus.cc:2258] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:40.130633 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 6855
I20260812 03:51:40.130764 7543 tablet_replica.cc:330] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: stopping tablet replica
I20260812 03:51:40.130844 7543 raft_consensus.cc:2229] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:51:40.130930 7543 raft_consensus.cc:2258] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:51:40.131719 7546 ts_tablet_manager.cc:1905] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:40.133222 7546 ts_tablet_manager.cc:1918] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:40.133282 7546 log.cc:1196] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-1/wal/wals/365415276f014f1ca93d9a3b9809b47a
I20260812 03:51:40.133468 7546 ts_tablet_manager.cc:1939] T 365415276f014f1ca93d9a3b9809b47a P 94fd6cb095974990aaea469de773db8b: Deleting consensus metadata
I20260812 03:51:40.133847 6731 catalog_manager.cc:4841] TS 94fd6cb095974990aaea469de773db8b (127.7.132.130:37301): tablet 365415276f014f1ca93d9a3b9809b47a (table test-table [id=62d8d0ef9d20416f8b91110394344af3]) successfully deleted
I20260812 03:51:40.138258 7543 ts_tablet_manager.cc:1905] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:51:40.139482 7543 ts_tablet_manager.cc:1918] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:51:40.139569 7543 log.cc:1196] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: Deleting WAL directory at /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786506457253237-7698-0/minicluster-data/ts-2/wal/wals/365415276f014f1ca93d9a3b9809b47a
I20260812 03:51:40.139799 7543 ts_tablet_manager.cc:1939] T 365415276f014f1ca93d9a3b9809b47a P 41f69e8d2e0e41d98fa253cea2b644fd: Deleting consensus metadata
I20260812 03:51:40.140192 6732 catalog_manager.cc:4841] TS 41f69e8d2e0e41d98fa253cea2b644fd (127.7.132.131:43911): tablet 365415276f014f1ca93d9a3b9809b47a (table test-table [id=62d8d0ef9d20416f8b91110394344af3]) successfully deleted
W20260812 03:51:40.143656 6733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.132.129:45249 (error 108)
I20260812 03:51:40.144089 7498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:40.143966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.129:45249 (local address 127.0.0.1:48338)
0812 03:51:40.144017 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:51:40.144059 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 03:51:40.144187 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 7063
W20260812 03:51:40.144191 6733 catalog_manager.cc:4584] TS c726bef1ed38498d8b064234865f479a (127.7.132.129:45249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 365415276f014f1ca93d9a3b9809b47a: Network error: Client connection negotiation failed: client connection to 127.7.132.129:45249: connect: Connection refused (error 111)
I20260812 03:51:40.151367 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 7279
I20260812 03:51:40.161746 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 6705
2026-08-12T03:51:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (705 ms)
[----------] 6 tests from EncryptionPolicyTest (8543 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:40 dist-test-slave-39l8 krb5kdc[7552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:40 dist-test-slave-39l8 krb5kdc[7552](info): set up 2 sockets
Aug 12 03:51:40 dist-test-slave-39l8 krb5kdc[7552](info): commencing operation
krb5kdc: starting...
W20260812 03:51:41.335789 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:41 dist-test-slave-39l8 krb5kdc[7552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:41.442334 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:41.518783 7571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:41.518944 7571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:41.518963 7571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:41.519927 7571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:41.519959 7571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:41.519973 7571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:41.519984 7571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:41.519994 7571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:41.521358 7571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:41.521545 7571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:41.521718 7571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:41.526122 7580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:41.526122 7576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:41.526221 7571 server_base.cc:1034] running on GCE node
W20260812 03:51:41.526221 7578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:41.526590 7571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:41.526916 7571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:41.528095 7571 hybrid_clock.cc:648] HybridClock initialized: now 1786506701528078 us; error 39 us; skew 500 ppm
Aug 12 03:51:41 dist-test-slave-39l8 krb5kdc[7552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506701, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:41.534929 7571 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:41.535336 7571 webserver.cc:460] Webserver started at http://127.0.0.1:34455/ using document root <none> and password file <none>
I20260812 03:51:41.535562 7571 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:41.535622 7571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:41.535741 7571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:41.536687 7571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "63cb848f83ea4e8497e49d312eb5e6bb"
format_stamp: "Formatted at 2026-08-12 03:51:41 on dist-test-slave-39l8"
server_key: "8049e1896371e935b1311f8c9b045ad9"
server_key_iv: "3a6667b517dc47d848af4e5ebcd951e9"
server_key_version: "encryptionkey@0"
I20260812 03:51:41.537007 7571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "63cb848f83ea4e8497e49d312eb5e6bb"
format_stamp: "Formatted at 2026-08-12 03:51:41 on dist-test-slave-39l8"
server_key: "8049e1896371e935b1311f8c9b045ad9"
server_key_iv: "3a6667b517dc47d848af4e5ebcd951e9"
server_key_version: "encryptionkey@0"
I20260812 03:51:41.538159 7571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:51:41.538791 7591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:41.538959 7571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:41.539047 7571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "63cb848f83ea4e8497e49d312eb5e6bb"
format_stamp: "Formatted at 2026-08-12 03:51:41 on dist-test-slave-39l8"
server_key: "8049e1896371e935b1311f8c9b045ad9"
server_key_iv: "3a6667b517dc47d848af4e5ebcd951e9"
server_key_version: "encryptionkey@0"
I20260812 03:51:41.539129 7571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:41.552212 7571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:41.553809 7571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:41.553956 7571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:41.557965 7571 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37051
I20260812 03:51:41.557994 7692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37051 every 8 connection(s)
I20260812 03:51:41.558300 7571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:41.559242 7693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:41.561313 7693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb: Bootstrap starting.
I20260812 03:51:41.561872 7693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:41.562157 7693 log.cc:824] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:41.562852 7693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb: No bootstrap required, opened a new log
I20260812 03:51:41.564293 7693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cb848f83ea4e8497e49d312eb5e6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37051 } }
I20260812 03:51:41.564430 7693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:41.564476 7693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63cb848f83ea4e8497e49d312eb5e6bb, State: Initialized, Role: FOLLOWER
I20260812 03:51:41.564634 7693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cb848f83ea4e8497e49d312eb5e6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37051 } }
I20260812 03:51:41.564699 7693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:41.564754 7693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:41.564802 7693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:41.565301 7693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cb848f83ea4e8497e49d312eb5e6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37051 } }
I20260812 03:51:41.565409 7693 leader_election.cc:302] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63cb848f83ea4e8497e49d312eb5e6bb; no voters:
I20260812 03:51:41.565583 7693 leader_election.cc:288] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:41.565696 7706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:41.565811 7693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:41.565855 7706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [term 1 LEADER]: Becoming Leader. State: Replica: 63cb848f83ea4e8497e49d312eb5e6bb, State: Running, Role: LEADER
I20260812 03:51:41.565928 7706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cb848f83ea4e8497e49d312eb5e6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37051 } }
I20260812 03:51:41.566366 7709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63cb848f83ea4e8497e49d312eb5e6bb. Latest consensus state: current_term: 1 leader_uuid: "63cb848f83ea4e8497e49d312eb5e6bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cb848f83ea4e8497e49d312eb5e6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37051 } } }
I20260812 03:51:41.566450 7709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:41.566567 7708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63cb848f83ea4e8497e49d312eb5e6bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cb848f83ea4e8497e49d312eb5e6bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37051 } } }
I20260812 03:51:41.566728 7708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:41.566816 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 7571
I20260812 03:51:41.566941 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:41.567052 7723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:41.567131 7698 external_mini_cluster.cc:1428] Setting key aa63cba3495bc31f9b1b35a6b12e70f3
I20260812 03:51:41.567553 7723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:41.568892 7723 catalog_manager.cc:1347] Generated new cluster ID: bc978ebf45eb43fb9ba18b35f40dd714
I20260812 03:51:41.568943 7723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:51:41 dist-test-slave-39l8 krb5kdc[7552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:51:41.577042 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:41.567716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34431 (local address 127.0.0.1:37051)
0812 03:51:41.568136 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:41.568143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:41.568176 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:51:41.568289 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:41.568294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:41.568349 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:41.568474 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:41.569196 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:41.569993 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:41.570870 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:41.571298 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:41.571355 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:41.573797 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:41.573800 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:41.573801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:41.573829 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:41.575454 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:41.575838 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:41.575841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:41.575841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:41.575966 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:41.576228 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:41.576229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:41.576230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:41.576446 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:41.576628 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:41.576724 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:51:41.576824 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":71,"threads_started":1}
I20260812 03:51:41.577211 7723 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:41.577791 7723 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:41.577911 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:41.585350 7723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63cb848f83ea4e8497e49d312eb5e6bb: Generated new TSK 0
I20260812 03:51:41.585497 7723 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:41.585567 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:41.579361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58633 (local address 127.0.0.1:37051)
0812 03:51:41.579395 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:41.579397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:41.579609 (+ 212us) server_negotiation.cc:407] Connection header received
0812 03:51:41.579655 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:41.579655 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:41.579666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:41.579721 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:41.580231 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:41.580747 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:41.581464 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:41.581719 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:41.581794 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:41.582937 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:41.582939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:41.582939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:41.582946 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:41.584377 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:41.584711 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:41.584713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:41.584713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:41.584763 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:41.584979 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:41.584981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:41.584981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:41.585100 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:41.585201 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:41.585386 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:51:41.585442 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 03:51:41.587653 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 7571
W20260812 03:51:41.602063 7548 connection.cc:535] client connection to 127.0.0.1:37051 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37051): Connection reset by peer (error 104)
2026-08-12T03:51:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:41 dist-test-slave-39l8 krb5kdc[7754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:41 dist-test-slave-39l8 krb5kdc[7754](info): set up 2 sockets
Aug 12 03:51:41 dist-test-slave-39l8 krb5kdc[7754](info): commencing operation
krb5kdc: starting...
W20260812 03:51:42.764874 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:42 dist-test-slave-39l8 krb5kdc[7754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:42.877537 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46789
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:42.955228 7776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:42.955421 7776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:42.955443 7776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:42.956545 7776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:42.956591 7776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:42.956605 7776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:42.956619 7776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:42.956630 7776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:42.958024 7776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46789
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:42.958216 7776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:42.958377 7776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:42.963215 7781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:42.963222 7784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:42.963222 7782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:42.963320 7776 server_base.cc:1034] running on GCE node
I20260812 03:51:42.963780 7776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:42.964033 7776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:42.965284 7776 hybrid_clock.cc:648] HybridClock initialized: now 1786506702965260 us; error 37 us; skew 500 ppm
Aug 12 03:51:42 dist-test-slave-39l8 krb5kdc[7754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506702, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:42.967758 7776 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:42.968199 7776 webserver.cc:460] Webserver started at http://127.0.0.1:33337/ using document root <none> and password file <none>
I20260812 03:51:42.968447 7776 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:42.968489 7776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:42.968649 7776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:42.969615 7776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "d762254c949d41df90979fe92e6bd3aa"
format_stamp: "Formatted at 2026-08-12 03:51:42 on dist-test-slave-39l8"
server_key: "f9efaf26a3a3e55c1f961ea7a35205db"
server_key_iv: "1bf8079ce9e2b7568c0b8e7331ae75af"
server_key_version: "encryptionkey@0"
I20260812 03:51:42.969960 7776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "d762254c949d41df90979fe92e6bd3aa"
format_stamp: "Formatted at 2026-08-12 03:51:42 on dist-test-slave-39l8"
server_key: "f9efaf26a3a3e55c1f961ea7a35205db"
server_key_iv: "1bf8079ce9e2b7568c0b8e7331ae75af"
server_key_version: "encryptionkey@0"
I20260812 03:51:42.971393 7776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:51:42.972138 7795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:42.972338 7776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:42.972435 7776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "d762254c949d41df90979fe92e6bd3aa"
format_stamp: "Formatted at 2026-08-12 03:51:42 on dist-test-slave-39l8"
server_key: "f9efaf26a3a3e55c1f961ea7a35205db"
server_key_iv: "1bf8079ce9e2b7568c0b8e7331ae75af"
server_key_version: "encryptionkey@0"
I20260812 03:51:42.972504 7776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:42.987216 7776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:42.988819 7776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:42.988965 7776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:42.992893 7776 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46789
I20260812 03:51:42.992944 7882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46789 every 8 connection(s)
I20260812 03:51:42.993266 7776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:42.994181 7886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:42.997021 7886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa: Bootstrap starting.
I20260812 03:51:42.997599 7886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:42.997880 7886 log.cc:824] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:42.998574 7886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa: No bootstrap required, opened a new log
I20260812 03:51:43.000133 7886 raft_consensus.cc:348] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d762254c949d41df90979fe92e6bd3aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } }
I20260812 03:51:43.000268 7886 raft_consensus.cc:374] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:43.000313 7886 raft_consensus.cc:729] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d762254c949d41df90979fe92e6bd3aa, State: Initialized, Role: FOLLOWER
I20260812 03:51:43.000454 7886 consensus_queue.cc:260] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d762254c949d41df90979fe92e6bd3aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } }
I20260812 03:51:43.000537 7886 raft_consensus.cc:388] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:43.000559 7886 raft_consensus.cc:482] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:43.000624 7886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:43.001138 7886 raft_consensus.cc:504] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d762254c949d41df90979fe92e6bd3aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } }
I20260812 03:51:43.001255 7886 leader_election.cc:302] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d762254c949d41df90979fe92e6bd3aa; no voters:
I20260812 03:51:43.001456 7886 leader_election.cc:288] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:43.001503 7889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:43.001701 7889 raft_consensus.cc:686] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [term 1 LEADER]: Becoming Leader. State: Replica: d762254c949d41df90979fe92e6bd3aa, State: Running, Role: LEADER
I20260812 03:51:43.001753 7886 sys_catalog.cc:564] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:43.001889 7889 consensus_queue.cc:237] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d762254c949d41df90979fe92e6bd3aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } }
I20260812 03:51:43.002036 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 7776
I20260812 03:51:43.002167 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:43.002388 7698 external_mini_cluster.cc:1428] Setting key d3c5850c8989cf7635bc348d89782ff1
I20260812 03:51:43.002663 7890 sys_catalog.cc:455] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d762254c949d41df90979fe92e6bd3aa. Latest consensus state: current_term: 1 leader_uuid: "d762254c949d41df90979fe92e6bd3aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d762254c949d41df90979fe92e6bd3aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } } }
I20260812 03:51:43.002789 7890 sys_catalog.cc:458] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:43.003139 7899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:43.003468 7891 sys_catalog.cc:455] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d762254c949d41df90979fe92e6bd3aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d762254c949d41df90979fe92e6bd3aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46789 } } }
I20260812 03:51:43.003571 7891 sys_catalog.cc:458] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:43.004122 7899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:43.007047 7899 catalog_manager.cc:1347] Generated new cluster ID: f96fea5b672b476293f7c91ea030666c
I20260812 03:51:43.007122 7899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:43.009016 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:43.003098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39815 (local address 127.0.0.1:46789)
0812 03:51:43.003298 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:43.003308 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:43.003341 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:51:43.003461 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:43.003469 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:43.003539 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:43.003965 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:43.004062 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:43.005567 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:43.006762 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:43.007390 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:43.007467 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:43.008350 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:43.008353 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:43.008354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:43.008541 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:43.008574 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:43.008587 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:43.008633 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:43.008794 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:51:43.008821 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260812 03:51:43.010203 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:43.015089 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:43.011711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53367 (local address 127.0.0.1:46789)
0812 03:51:43.011742 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:43.011744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:43.011810 (+ 66us) server_negotiation.cc:407] Connection header received
0812 03:51:43.011910 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:43.011911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:43.011920 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:43.011966 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:43.012133 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:43.012829 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:43.013806 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:43.014099 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:43.014149 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:43.014856 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:43.014857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:43.014857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:43.014861 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:43.014875 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:43.014879 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:43.014928 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:43.015016 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:51:43.015027 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 03:51:43.015938 7899 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:43.016367 7899 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:43.018682 7899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d762254c949d41df90979fe92e6bd3aa: Generated new TSK 0
I20260812 03:51:43.018847 7899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:43.025830 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 7776
W20260812 03:51:43.048801 7751 connection.cc:535] client connection to 127.0.0.1:46789 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46789): Connection reset by peer (error 104)
2026-08-12T03:51:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:43 dist-test-slave-39l8 krb5kdc[7927](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:43 dist-test-slave-39l8 krb5kdc[7927](info): set up 2 sockets
Aug 12 03:51:43 dist-test-slave-39l8 krb5kdc[7927](info): commencing operation
krb5kdc: starting...
W20260812 03:51:44.214411 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:44 dist-test-slave-39l8 krb5kdc[7927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:44.325390 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42943
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:44.401684 7951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:44.401925 7951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:44.401960 7951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:44.403067 7951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:44.403126 7951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:44.403157 7951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:44.403185 7951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:44.403208 7951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:44.404670 7951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42943
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:44.404953 7951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:44.405182 7951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:44.410004 7959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:44.409987 7963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:44.410014 7951 server_base.cc:1034] running on GCE node
W20260812 03:51:44.409986 7960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:44.410431 7951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:44.410790 7951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:44.411957 7951 hybrid_clock.cc:648] HybridClock initialized: now 1786506704411934 us; error 38 us; skew 500 ppm
Aug 12 03:51:44 dist-test-slave-39l8 krb5kdc[7927](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506704, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:44.414301 7951 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:44.414680 7951 webserver.cc:460] Webserver started at http://127.0.0.1:45397/ using document root <none> and password file <none>
I20260812 03:51:44.414917 7951 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:44.414994 7951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:44.415112 7951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:44.416333 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "912cba6037aa4b44a6031251bf9bd4ed"
format_stamp: "Formatted at 2026-08-12 03:51:44 on dist-test-slave-39l8"
server_key: "9ebbfe4bffcede5f6542bd6d4157cd33"
server_key_iv: "150f7fdb1bf1db66ce65e5bb3e93966d"
server_key_version: "encryptionkey@0"
I20260812 03:51:44.416616 7951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "912cba6037aa4b44a6031251bf9bd4ed"
format_stamp: "Formatted at 2026-08-12 03:51:44 on dist-test-slave-39l8"
server_key: "9ebbfe4bffcede5f6542bd6d4157cd33"
server_key_iv: "150f7fdb1bf1db66ce65e5bb3e93966d"
server_key_version: "encryptionkey@0"
I20260812 03:51:44.417788 7951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:51:44.418413 7972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:44.418570 7951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:44.418670 7951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "912cba6037aa4b44a6031251bf9bd4ed"
format_stamp: "Formatted at 2026-08-12 03:51:44 on dist-test-slave-39l8"
server_key: "9ebbfe4bffcede5f6542bd6d4157cd33"
server_key_iv: "150f7fdb1bf1db66ce65e5bb3e93966d"
server_key_version: "encryptionkey@0"
I20260812 03:51:44.418762 7951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:44.421922 7951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:44.423688 7951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:44.423831 7951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:44.427621 7951 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42943
I20260812 03:51:44.427655 8056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42943 every 8 connection(s)
I20260812 03:51:44.427968 7951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:44.428951 8058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:44.429814 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 7951
I20260812 03:51:44.429950 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:44.430148 7698 external_mini_cluster.cc:1428] Setting key b491d461d5e4f4754f6897476b7de719
I20260812 03:51:44.432296 8058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed: Bootstrap starting.
I20260812 03:51:44.432451 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:44.430647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40133 (local address 127.0.0.1:42943)
0812 03:51:44.430853 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:44.430861 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:44.430893 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:51:44.431291 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:44.431297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:44.431321 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:44.431426 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:44.432015 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:44.432018 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:44.432018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:44.432021 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:44.432065 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:44.432074 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:44.432123 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:44.432210 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 03:51:44.432260 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":122,"threads_started":1}
I20260812 03:51:44.433004 8058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:44.433310 8058 log.cc:824] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:44.433465 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:44.433967 8058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed: No bootstrap required, opened a new log
I20260812 03:51:44.435572 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:44.434777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48385 (local address 127.0.0.1:42943)
0812 03:51:44.434812 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:44.434813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:44.434846 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:51:44.434909 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:44.434910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:44.434913 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:44.434951 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:44.435394 (+ 443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:44.435395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:44.435395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:44.435398 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:44.435408 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:44.435411 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:44.435446 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:44.435493 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 03:51:44.435502 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:51:44.436381 8058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cba6037aa4b44a6031251bf9bd4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42943 } }
I20260812 03:51:44.436508 8058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:44.436529 8058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912cba6037aa4b44a6031251bf9bd4ed, State: Initialized, Role: FOLLOWER
I20260812 03:51:44.436656 8058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cba6037aa4b44a6031251bf9bd4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42943 } }
I20260812 03:51:44.436743 8058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:44.436782 8058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:44.436827 8058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:44.437305 8058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cba6037aa4b44a6031251bf9bd4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42943 } }
I20260812 03:51:44.437413 8058 leader_election.cc:302] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 912cba6037aa4b44a6031251bf9bd4ed; no voters:
I20260812 03:51:44.437582 8058 leader_election.cc:288] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:44.437630 8074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:44.437810 8058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:44.437809 8074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [term 1 LEADER]: Becoming Leader. State: Replica: 912cba6037aa4b44a6031251bf9bd4ed, State: Running, Role: LEADER
I20260812 03:51:44.437934 8074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cba6037aa4b44a6031251bf9bd4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42943 } }
I20260812 03:51:44.438302 8075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "912cba6037aa4b44a6031251bf9bd4ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cba6037aa4b44a6031251bf9bd4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42943 } } }
I20260812 03:51:44.438385 8075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:44.438591 8087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:44.438581 8077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 912cba6037aa4b44a6031251bf9bd4ed. Latest consensus state: current_term: 1 leader_uuid: "912cba6037aa4b44a6031251bf9bd4ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cba6037aa4b44a6031251bf9bd4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42943 } } }
I20260812 03:51:44.438737 8077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:44.439136 8087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:44.440416 8087 catalog_manager.cc:1347] Generated new cluster ID: 53b269ab880e44b6b013684b76fb0a48
I20260812 03:51:44.440467 8087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:44.451812 8087 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:44.451958 8087 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:44.454231 8087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 912cba6037aa4b44a6031251bf9bd4ed: Generated new TSK 0
I20260812 03:51:44.454375 8087 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:44.460039 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 7951
2026-08-12T03:51:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:44 dist-test-slave-39l8 krb5kdc[8108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:44 dist-test-slave-39l8 krb5kdc[8108](info): set up 2 sockets
Aug 12 03:51:44 dist-test-slave-39l8 krb5kdc[8108](info): commencing operation
krb5kdc: starting...
W20260812 03:51:45.637527 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:45 dist-test-slave-39l8 krb5kdc[8108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:45.748679 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:45.826515 8130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:45.826692 8130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:45.826728 8130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:45.827802 8130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:45.827842 8130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:45.827868 8130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:45.827919 8130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:45.827942 8130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:45.829273 8130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:45.829495 8130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:45.829715 8130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:45.834394 8139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:45.834431 8138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:45.834434 8144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:45.834589 8130 server_base.cc:1034] running on GCE node
I20260812 03:51:45.834833 8130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:45.835108 8130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:45.836272 8130 hybrid_clock.cc:648] HybridClock initialized: now 1786506705836258 us; error 47 us; skew 500 ppm
Aug 12 03:51:45 dist-test-slave-39l8 krb5kdc[8108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:45.838686 8130 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:45.839195 8130 webserver.cc:460] Webserver started at http://127.0.0.1:36761/ using document root <none> and password file <none>
I20260812 03:51:45.839423 8130 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:45.839490 8130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:45.839643 8130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:45.840618 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "2020515a269d40058149cf7c538012ac"
format_stamp: "Formatted at 2026-08-12 03:51:45 on dist-test-slave-39l8"
server_key: "ebf2bcf9d65d5bdd2de7007ea716ad42"
server_key_iv: "15956a671e5f12838cdde82f63fbf5b1"
server_key_version: "encryptionkey@0"
I20260812 03:51:45.841007 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "2020515a269d40058149cf7c538012ac"
format_stamp: "Formatted at 2026-08-12 03:51:45 on dist-test-slave-39l8"
server_key: "ebf2bcf9d65d5bdd2de7007ea716ad42"
server_key_iv: "15956a671e5f12838cdde82f63fbf5b1"
server_key_version: "encryptionkey@0"
I20260812 03:51:45.842322 8130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:51:45.843145 8154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:45.843330 8130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:45.843431 8130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "2020515a269d40058149cf7c538012ac"
format_stamp: "Formatted at 2026-08-12 03:51:45 on dist-test-slave-39l8"
server_key: "ebf2bcf9d65d5bdd2de7007ea716ad42"
server_key_iv: "15956a671e5f12838cdde82f63fbf5b1"
server_key_version: "encryptionkey@0"
I20260812 03:51:45.843523 8130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:45.860908 8130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:45.863188 8130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:45.863356 8130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:45.867551 8130 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43027
I20260812 03:51:45.867604 8242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43027 every 8 connection(s)
I20260812 03:51:45.867949 8130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:45.868882 8243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:45.871115 8243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac: Bootstrap starting.
I20260812 03:51:45.871667 8243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:45.871905 8243 log.cc:824] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:45.872556 8243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac: No bootstrap required, opened a new log
I20260812 03:51:45.873337 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8130
I20260812 03:51:45.873468 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:45.873648 7698 external_mini_cluster.cc:1428] Setting key c1d896d3fc7771f707cd2a548d3c8768
I20260812 03:51:45.875042 8243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2020515a269d40058149cf7c538012ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43027 } }
I20260812 03:51:45.875279 8243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:45.875352 8243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2020515a269d40058149cf7c538012ac, State: Initialized, Role: FOLLOWER
I20260812 03:51:45.875554 8243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2020515a269d40058149cf7c538012ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43027 } }
I20260812 03:51:45.875648 8243 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:45.875686 8243 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:45.875749 8243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:45.876374 8243 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2020515a269d40058149cf7c538012ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43027 } }
I20260812 03:51:45.876479 8243 leader_election.cc:302] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2020515a269d40058149cf7c538012ac; no voters:
I20260812 03:51:45.876641 8243 leader_election.cc:288] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:45.876700 8254 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:45.876853 8254 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [term 1 LEADER]: Becoming Leader. State: Replica: 2020515a269d40058149cf7c538012ac, State: Running, Role: LEADER
I20260812 03:51:45.876914 8243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:45.876977 8254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2020515a269d40058149cf7c538012ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43027 } }
I20260812 03:51:45.877404 8255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2020515a269d40058149cf7c538012ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2020515a269d40058149cf7c538012ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43027 } } }
I20260812 03:51:45.877483 8255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:45.877697 8256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2020515a269d40058149cf7c538012ac. Latest consensus state: current_term: 1 leader_uuid: "2020515a269d40058149cf7c538012ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2020515a269d40058149cf7c538012ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43027 } } }
I20260812 03:51:45.877817 8256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:45.877985 8267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:45.878537 8267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:45.880338 8267 catalog_manager.cc:1347] Generated new cluster ID: 4f10ab12f8894a7582063bcb1ce393c1
I20260812 03:51:45.880398 8267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:51:45 dist-test-slave-39l8 krb5kdc[8108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:51:45.886541 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:45.874155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52251 (local address 127.0.0.1:43027)
0812 03:51:45.874328 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:45.874343 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:45.874367 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:51:45.875221 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:45.875229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:45.875290 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:45.875433 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:45.876265 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:45.877609 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:45.878822 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:45.879225 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:45.879341 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:45.882204 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:45.882208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:45.882208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:45.882220 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:45.884312 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:45.884795 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:45.884800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:45.884800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:45.884932 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:45.885188 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:45.885189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:45.885189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:45.885524 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:45.885680 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:45.886188 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 03:51:45.886336 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260812 03:51:45.887493 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:45.892499 8267 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:45.892956 8267 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:45.896328 8267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2020515a269d40058149cf7c538012ac: Generated new TSK 0
I20260812 03:51:45.896500 8267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:45.897684 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:45.889039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54619 (local address 127.0.0.1:43027)
0812 03:51:45.889073 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:45.889074 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:45.890868 (+ 1794us) server_negotiation.cc:407] Connection header received
0812 03:51:45.890900 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:45.890901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:45.890912 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:45.890979 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:45.891606 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:45.892431 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:45.893412 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:45.893702 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:45.893766 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:45.895088 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:45.895090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:45.895090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:45.895096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:45.896688 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:45.897033 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:45.897033 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:45.897033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:45.897077 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:45.897269 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:45.897269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:45.897270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:45.897371 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:45.897463 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:45.897578 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 03:51:45.897636 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260812 03:51:45.899354 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8130
2026-08-12T03:51:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:45 dist-test-slave-39l8 krb5kdc[8294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:45 dist-test-slave-39l8 krb5kdc[8294](info): set up 2 sockets
Aug 12 03:51:45 dist-test-slave-39l8 krb5kdc[8294](info): commencing operation
krb5kdc: starting...
W20260812 03:51:47.076655 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:47 dist-test-slave-39l8 krb5kdc[8294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:47.189610 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35719
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:47.267557 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:47.267740 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:47.267761 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:47.268787 8317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:47.268826 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:47.268838 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:47.268875 8317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:47.268903 8317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:47.270257 8317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:47.270495 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:47.270699 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:47.275344 8328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:47.275370 8317 server_base.cc:1034] running on GCE node
W20260812 03:51:47.275345 8326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:47.275370 8333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:47.275707 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:47.276007 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:47.277156 8317 hybrid_clock.cc:648] HybridClock initialized: now 1786506707277131 us; error 43 us; skew 500 ppm
Aug 12 03:51:47 dist-test-slave-39l8 krb5kdc[8294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506707, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:47.284464 8317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:47.284844 8317 webserver.cc:460] Webserver started at http://127.0.0.1:42961/ using document root <none> and password file <none>
I20260812 03:51:47.285066 8317 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:47.285110 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:47.285277 8317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:47.286267 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "2e8ea9fcc53642a18944a028da59661a"
format_stamp: "Formatted at 2026-08-12 03:51:47 on dist-test-slave-39l8"
server_key: "7ba2f5c5084e779ba6bd51a330eb98b4"
server_key_iv: "1fc5704cdbc1f4e38c452efce1279787"
server_key_version: "encryptionkey@0"
I20260812 03:51:47.286700 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "2e8ea9fcc53642a18944a028da59661a"
format_stamp: "Formatted at 2026-08-12 03:51:47 on dist-test-slave-39l8"
server_key: "7ba2f5c5084e779ba6bd51a330eb98b4"
server_key_iv: "1fc5704cdbc1f4e38c452efce1279787"
server_key_version: "encryptionkey@0"
I20260812 03:51:47.287946 8317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:51:47.288601 8341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:47.288807 8317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:47.288906 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "2e8ea9fcc53642a18944a028da59661a"
format_stamp: "Formatted at 2026-08-12 03:51:47 on dist-test-slave-39l8"
server_key: "7ba2f5c5084e779ba6bd51a330eb98b4"
server_key_iv: "1fc5704cdbc1f4e38c452efce1279787"
server_key_version: "encryptionkey@0"
I20260812 03:51:47.288986 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:47.303550 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:47.305222 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:47.305383 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:47.309531 8317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35719
I20260812 03:51:47.309568 8427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35719 every 8 connection(s)
I20260812 03:51:47.309922 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:47.310981 8428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:47.313215 8428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a: Bootstrap starting.
I20260812 03:51:47.313781 8428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:47.314054 8428 log.cc:824] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:47.314265 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8317
I20260812 03:51:47.314393 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:47.314550 7698 external_mini_cluster.cc:1428] Setting key 5188dfef22645db18c977b891ac1b29e
I20260812 03:51:47.314893 8428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a: No bootstrap required, opened a new log
I20260812 03:51:47.317611 8428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8ea9fcc53642a18944a028da59661a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35719 } }
I20260812 03:51:47.317773 8428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:47.317802 8428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8ea9fcc53642a18944a028da59661a, State: Initialized, Role: FOLLOWER
I20260812 03:51:47.317902 8428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8ea9fcc53642a18944a028da59661a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35719 } }
I20260812 03:51:47.317970 8428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:47.318003 8428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:47.318056 8428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:47.318542 8428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8ea9fcc53642a18944a028da59661a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35719 } }
I20260812 03:51:47.318665 8428 leader_election.cc:302] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e8ea9fcc53642a18944a028da59661a; no voters:
I20260812 03:51:47.318882 8428 leader_election.cc:288] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:47.318907 8435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:47.319012 8435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [term 1 LEADER]: Becoming Leader. State: Replica: 2e8ea9fcc53642a18944a028da59661a, State: Running, Role: LEADER
I20260812 03:51:47.319161 8428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:47.319139 8435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8ea9fcc53642a18944a028da59661a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35719 } }
I20260812 03:51:47.319766 8436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e8ea9fcc53642a18944a028da59661a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8ea9fcc53642a18944a028da59661a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35719 } } }
I20260812 03:51:47.320005 8436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:47.319927 8437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e8ea9fcc53642a18944a028da59661a. Latest consensus state: current_term: 1 leader_uuid: "2e8ea9fcc53642a18944a028da59661a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8ea9fcc53642a18944a028da59661a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35719 } } }
I20260812 03:51:47.320216 8451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:47.320223 8437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:47.320750 8451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:47.320782 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:47.315060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48667 (local address 127.0.0.1:35719)
0812 03:51:47.315228 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:47.315237 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:47.315267 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:51:47.316278 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:47.316287 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:47.316344 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:47.316485 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:47.316631 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:47.317929 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:47.318684 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:47.319251 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:47.319343 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:47.320081 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:47.320085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:47.320086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:47.320293 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:47.320337 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:47.320353 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:47.320418 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:47.320534 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 03:51:47.320594 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260812 03:51:47.321714 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:47.327118 8451 catalog_manager.cc:1347] Generated new cluster ID: 2dc44261c1324848b46417296d8d8fbd
I20260812 03:51:47.327178 8451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:47.329849 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:47.327271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34103 (local address 127.0.0.1:35719)
0812 03:51:47.327305 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:47.327307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:47.327523 (+ 216us) server_negotiation.cc:407] Connection header received
0812 03:51:47.327580 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:47.327581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:47.327590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:47.327641 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:47.327766 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:47.328174 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:47.328830 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:47.329031 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:47.329100 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:47.329633 (+ 533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:47.329634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:47.329635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:47.329638 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:47.329657 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:47.329660 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:47.329703 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:47.329754 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:51:47.329772 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:51:47.334336 8451 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:47.334887 8451 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:47.345217 8451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e8ea9fcc53642a18944a028da59661a: Generated new TSK 0
I20260812 03:51:47.345381 8451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:47.348582 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8317
2026-08-12T03:51:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:47 dist-test-slave-39l8 krb5kdc[8474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:47 dist-test-slave-39l8 krb5kdc[8474](info): set up 2 sockets
Aug 12 03:51:47 dist-test-slave-39l8 krb5kdc[8474](info): commencing operation
krb5kdc: starting...
W20260812 03:51:48.539958 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:48 dist-test-slave-39l8 krb5kdc[8474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:48.649295 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:48.728787 8497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:48.728997 8497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:48.729034 8497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:48.730098 8497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:48.730158 8497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:48.730180 8497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:48.730207 8497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:48.730232 8497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:48.731750 8497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:48.732020 8497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:48.732283 8497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:48.736949 8506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:48.736991 8505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:48.737126 8497 server_base.cc:1034] running on GCE node
W20260812 03:51:48.737171 8511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:48.737402 8497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:48.737697 8497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:48.738862 8497 hybrid_clock.cc:648] HybridClock initialized: now 1786506708738851 us; error 28 us; skew 500 ppm
Aug 12 03:51:48 dist-test-slave-39l8 krb5kdc[8474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506708, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:48.741192 8497 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:48.741566 8497 webserver.cc:460] Webserver started at http://127.0.0.1:33603/ using document root <none> and password file <none>
I20260812 03:51:48.741772 8497 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:48.741813 8497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:48.741899 8497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:48.742911 8497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "a2bbab715fb84384b67fa3b4734cd4d7"
format_stamp: "Formatted at 2026-08-12 03:51:48 on dist-test-slave-39l8"
server_key: "44581f52200ec07379c0b31e57adbbe6"
server_key_iv: "b87ffd3a83091c264a6ab92a642efa26"
server_key_version: "encryptionkey@0"
I20260812 03:51:48.743201 8497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "a2bbab715fb84384b67fa3b4734cd4d7"
format_stamp: "Formatted at 2026-08-12 03:51:48 on dist-test-slave-39l8"
server_key: "44581f52200ec07379c0b31e57adbbe6"
server_key_iv: "b87ffd3a83091c264a6ab92a642efa26"
server_key_version: "encryptionkey@0"
I20260812 03:51:48.744479 8497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:48.745110 8524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:48.745270 8497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:48.745352 8497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "a2bbab715fb84384b67fa3b4734cd4d7"
format_stamp: "Formatted at 2026-08-12 03:51:48 on dist-test-slave-39l8"
server_key: "44581f52200ec07379c0b31e57adbbe6"
server_key_iv: "b87ffd3a83091c264a6ab92a642efa26"
server_key_version: "encryptionkey@0"
I20260812 03:51:48.745404 8497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:48.748337 8497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:48.749972 8497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:48.750070 8497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:48.753887 8497 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34323
I20260812 03:51:48.753961 8618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34323 every 8 connection(s)
I20260812 03:51:48.754243 8497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:48.755213 8619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:48.757222 8619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7: Bootstrap starting.
I20260812 03:51:48.757754 8619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:48.758023 8619 log.cc:824] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:48.758669 8619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7: No bootstrap required, opened a new log
I20260812 03:51:48.760213 8619 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bbab715fb84384b67fa3b4734cd4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260812 03:51:48.760361 8619 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:48.760406 8619 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2bbab715fb84384b67fa3b4734cd4d7, State: Initialized, Role: FOLLOWER
I20260812 03:51:48.760555 8619 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bbab715fb84384b67fa3b4734cd4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260812 03:51:48.760660 8619 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:48.760704 8619 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:48.760757 8619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:48.761289 8619 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bbab715fb84384b67fa3b4734cd4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260812 03:51:48.761411 8619 leader_election.cc:302] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2bbab715fb84384b67fa3b4734cd4d7; no voters:
I20260812 03:51:48.761600 8619 leader_election.cc:288] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:48.761628 8623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:48.761793 8623 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [term 1 LEADER]: Becoming Leader. State: Replica: a2bbab715fb84384b67fa3b4734cd4d7, State: Running, Role: LEADER
I20260812 03:51:48.761878 8619 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:48.761956 8623 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bbab715fb84384b67fa3b4734cd4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } }
I20260812 03:51:48.762521 8624 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2bbab715fb84384b67fa3b4734cd4d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bbab715fb84384b67fa3b4734cd4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } } }
I20260812 03:51:48.762537 8625 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2bbab715fb84384b67fa3b4734cd4d7. Latest consensus state: current_term: 1 leader_uuid: "a2bbab715fb84384b67fa3b4734cd4d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2bbab715fb84384b67fa3b4734cd4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34323 } } }
I20260812 03:51:48.762635 8624 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:48.762662 8625 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:48.762849 8637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:48.763393 8637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:48.763832 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8497
I20260812 03:51:48.763947 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:48.764117 7698 external_mini_cluster.cc:1428] Setting key 6e7235780a24ea5953ea99347d8791cc
I20260812 03:51:48.766031 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:48.764614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43613 (local address 127.0.0.1:34323)
0812 03:51:48.764760 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:48.764769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:48.764797 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:51:48.764900 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:48.764906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:48.764932 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:48.765057 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:48.765654 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:48.765658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:48.765659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:48.765664 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:48.765701 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:48.765714 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:48.765787 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:48.765836 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 03:51:48.765873 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260812 03:51:48.767009 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:48.767091 8637 catalog_manager.cc:1347] Generated new cluster ID: 96a3adc50fa5429cbf268d0bb901b432
I20260812 03:51:48.767129 8637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:48.771855 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:48.768445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47461 (local address 127.0.0.1:34323)
0812 03:51:48.768477 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:48.768479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:48.770906 (+ 2427us) server_negotiation.cc:407] Connection header received
0812 03:51:48.770956 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:48.770957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:48.770963 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:48.771042 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:48.771633 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:48.771635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:48.771635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:48.771639 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:48.771654 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:48.771659 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:48.771723 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:48.771777 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:51:48.771791 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 03:51:48.773687 8637 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:48.773828 8637 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:48.777110 8637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2bbab715fb84384b67fa3b4734cd4d7: Generated new TSK 0
I20260812 03:51:48.777261 8637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:48.781579 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8497
2026-08-12T03:51:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:48 dist-test-slave-39l8 krb5kdc[8671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:48 dist-test-slave-39l8 krb5kdc[8671](info): set up 2 sockets
Aug 12 03:51:48 dist-test-slave-39l8 krb5kdc[8671](info): commencing operation
krb5kdc: starting...
W20260812 03:51:49.971396 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:50 dist-test-slave-39l8 krb5kdc[8671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:50.088861 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:50.165891 8693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:50.166114 8693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:50.166154 8693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:50.167371 8693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:50.167438 8693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:50.167462 8693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:50.167490 8693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:50.167519 8693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:50.168964 8693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:50.169272 8693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:50.169505 8693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:50.174185 8704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:50.174141 8708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:50.174152 8703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:50.174402 8693 server_base.cc:1034] running on GCE node
I20260812 03:51:50.174646 8693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:50.174973 8693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:50.176115 8693 hybrid_clock.cc:648] HybridClock initialized: now 1786506710176094 us; error 41 us; skew 500 ppm
Aug 12 03:51:50 dist-test-slave-39l8 krb5kdc[8671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506710, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:50.178507 8693 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:50.178973 8693 webserver.cc:460] Webserver started at http://127.0.0.1:42599/ using document root <none> and password file <none>
I20260812 03:51:50.179154 8693 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:50.179194 8693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:50.179278 8693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:50.180205 8693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "db15f09789db45ffb640d664e39a5648"
format_stamp: "Formatted at 2026-08-12 03:51:50 on dist-test-slave-39l8"
server_key: "f28ebd75352bfab609214d0773c4e239"
server_key_iv: "83a56336b490f28655c41ece22fc4a25"
server_key_version: "encryptionkey@0"
I20260812 03:51:50.180497 8693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "db15f09789db45ffb640d664e39a5648"
format_stamp: "Formatted at 2026-08-12 03:51:50 on dist-test-slave-39l8"
server_key: "f28ebd75352bfab609214d0773c4e239"
server_key_iv: "83a56336b490f28655c41ece22fc4a25"
server_key_version: "encryptionkey@0"
I20260812 03:51:50.181725 8693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:51:50.182440 8723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:50.182631 8693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:51:50.182765 8693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "db15f09789db45ffb640d664e39a5648"
format_stamp: "Formatted at 2026-08-12 03:51:50 on dist-test-slave-39l8"
server_key: "f28ebd75352bfab609214d0773c4e239"
server_key_iv: "83a56336b490f28655c41ece22fc4a25"
server_key_version: "encryptionkey@0"
I20260812 03:51:50.182854 8693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:50.209177 8693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:50.211313 8693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:50.211449 8693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:50.215329 8693 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32773
I20260812 03:51:50.215376 8809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32773 every 8 connection(s)
I20260812 03:51:50.215690 8693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:50.216629 8810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:50.218819 8810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648: Bootstrap starting.
I20260812 03:51:50.219355 8810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:50.219616 8810 log.cc:824] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:50.220232 8810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648: No bootstrap required, opened a new log
I20260812 03:51:50.221789 8810 raft_consensus.cc:348] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db15f09789db45ffb640d664e39a5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260812 03:51:50.221920 8810 raft_consensus.cc:374] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:50.221966 8810 raft_consensus.cc:729] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db15f09789db45ffb640d664e39a5648, State: Initialized, Role: FOLLOWER
I20260812 03:51:50.222117 8810 consensus_queue.cc:260] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db15f09789db45ffb640d664e39a5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260812 03:51:50.222182 8810 raft_consensus.cc:388] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:50.222203 8810 raft_consensus.cc:482] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:50.222268 8810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:50.222808 8810 raft_consensus.cc:504] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db15f09789db45ffb640d664e39a5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260812 03:51:50.222909 8810 leader_election.cc:302] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db15f09789db45ffb640d664e39a5648; no voters:
I20260812 03:51:50.223073 8810 leader_election.cc:288] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:50.223093 8814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:50.223305 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8693
I20260812 03:51:50.223308 8810 sys_catalog.cc:564] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:50.223413 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:50.223598 7698 external_mini_cluster.cc:1428] Setting key d8a4975f1f01d09c230b672d59eec813
I20260812 03:51:50.223821 8814 raft_consensus.cc:686] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [term 1 LEADER]: Becoming Leader. State: Replica: db15f09789db45ffb640d664e39a5648, State: Running, Role: LEADER
I20260812 03:51:50.223953 8814 consensus_queue.cc:237] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db15f09789db45ffb640d664e39a5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260812 03:51:50.224395 8821 sys_catalog.cc:455] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db15f09789db45ffb640d664e39a5648. Latest consensus state: current_term: 1 leader_uuid: "db15f09789db45ffb640d664e39a5648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db15f09789db45ffb640d664e39a5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } } }
I20260812 03:51:50.224558 8821 sys_catalog.cc:458] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:50.224534 8815 sys_catalog.cc:455] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db15f09789db45ffb640d664e39a5648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db15f09789db45ffb640d664e39a5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } } }
I20260812 03:51:50.224817 8815 sys_catalog.cc:458] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648 [sys.catalog]: This master's current role is: LEADER
W20260812 03:51:50.225494 8840 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:51:50.225550 8840 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:51:50.225601 8842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:50.225773 8842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:50.227025 8842 catalog_manager.cc:1347] Generated new cluster ID: 6d131edf8e134d3986a19601a411a455
I20260812 03:51:50.227077 8842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:51:50 dist-test-slave-39l8 krb5kdc[8671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:51:50.234261 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:50.224163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:47527 (local address 127.0.0.1:32773)
0812 03:51:50.224886 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:50.224896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:50.225076 (+ 180us) server_negotiation.cc:407] Connection header received
0812 03:51:50.225237 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:50.225244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:50.225295 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:50.225445 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:50.226167 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:50.227544 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:50.228266 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:50.228813 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:50.228933 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:50.231096 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:50.231100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:50.231124 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:50.231152 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:50.232799 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:50.233160 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:50.233169 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:50.233169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:50.233260 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:50.233473 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:50.233474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:50.233475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:50.233684 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:50.233908 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:50.234004 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:51:50.234104 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":116,"threads_started":1}
I20260812 03:51:50.235194 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:50.242374 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:50.236606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:50857 (local address 127.0.0.1:32773)
0812 03:51:50.236641 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:50.236643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:50.236767 (+ 124us) server_negotiation.cc:407] Connection header received
0812 03:51:50.236833 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:50.236833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:50.236842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:50.236882 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:50.237345 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:50.237775 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:50.238442 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:50.238656 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:50.238742 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:50.239856 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:50.239858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:50.239858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:50.239864 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:50.241183 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:50.241551 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:50.241552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:50.241553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:50.241613 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:50.241839 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:50.241840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:50.241840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:50.241947 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:50.242067 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:50.242184 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 03:51:50.242264 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:51:50.249893 8842 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:50.250329 8842 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:50.254637 8842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db15f09789db45ffb640d664e39a5648: Generated new TSK 0
I20260812 03:51:50.254827 8842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:50.263381 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8693
W20260812 03:51:50.274547 8663 connection.cc:535] client connection to 127.0.0.1:32773 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32773): Connection reset by peer (error 104)
2026-08-12T03:51:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:50 dist-test-slave-39l8 krb5kdc[8864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:50 dist-test-slave-39l8 krb5kdc[8864](info): set up 2 sockets
Aug 12 03:51:50 dist-test-slave-39l8 krb5kdc[8864](info): commencing operation
krb5kdc: starting...
W20260812 03:51:51.436167 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:51 dist-test-slave-39l8 krb5kdc[8864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:51.545222 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:51.621269 8887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:51.621470 8887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:51.621493 8887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:51.623050 8887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:51.623107 8887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:51.623129 8887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:51.623147 8887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:51.623172 8887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:51.624615 8887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:51.624861 8887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:51.625061 8887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:51.629328 8897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:51.629407 8894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:51.629534 8887 server_base.cc:1034] running on GCE node
W20260812 03:51:51.629568 8895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:51.629818 8887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:51.630127 8887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:51.631302 8887 hybrid_clock.cc:648] HybridClock initialized: now 1786506711631291 us; error 28 us; skew 500 ppm
Aug 12 03:51:51 dist-test-slave-39l8 krb5kdc[8864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506711, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:51.633734 8887 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:51.634075 8887 webserver.cc:460] Webserver started at http://127.0.0.1:43645/ using document root <none> and password file <none>
I20260812 03:51:51.634254 8887 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:51.634305 8887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:51.634380 8887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:51.635290 8887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "a58acb114e3c436c964e5e77fa807662"
format_stamp: "Formatted at 2026-08-12 03:51:51 on dist-test-slave-39l8"
server_key: "0c4e7bc543d3f53966e5c7c76c2fe8a3"
server_key_iv: "c16b7e429b537bff670b410daba36b50"
server_key_version: "encryptionkey@0"
I20260812 03:51:51.635564 8887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "a58acb114e3c436c964e5e77fa807662"
format_stamp: "Formatted at 2026-08-12 03:51:51 on dist-test-slave-39l8"
server_key: "0c4e7bc543d3f53966e5c7c76c2fe8a3"
server_key_iv: "c16b7e429b537bff670b410daba36b50"
server_key_version: "encryptionkey@0"
I20260812 03:51:51.637125 8887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:51:51.637733 8909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:51.637880 8887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:51.637960 8887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "a58acb114e3c436c964e5e77fa807662"
format_stamp: "Formatted at 2026-08-12 03:51:51 on dist-test-slave-39l8"
server_key: "0c4e7bc543d3f53966e5c7c76c2fe8a3"
server_key_iv: "c16b7e429b537bff670b410daba36b50"
server_key_version: "encryptionkey@0"
I20260812 03:51:51.638010 8887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:51.649258 8887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:51.650995 8887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:51.651098 8887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:51.654592 8887 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46101
I20260812 03:51:51.654636 9009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46101 every 8 connection(s)
I20260812 03:51:51.654954 8887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:51.655839 9010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:51.657716 9010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662: Bootstrap starting.
I20260812 03:51:51.658238 9010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:51.658500 9010 log.cc:824] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:51.659170 9010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662: No bootstrap required, opened a new log
I20260812 03:51:51.659627 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 8887
I20260812 03:51:51.659726 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:51.659922 7698 external_mini_cluster.cc:1428] Setting key 266451ef69f9df134ccfeded4605c289
I20260812 03:51:51.661598 9010 raft_consensus.cc:348] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58acb114e3c436c964e5e77fa807662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } }
I20260812 03:51:51.661772 9010 raft_consensus.cc:374] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:51.661813 9010 raft_consensus.cc:729] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58acb114e3c436c964e5e77fa807662, State: Initialized, Role: FOLLOWER
I20260812 03:51:51.661934 9010 consensus_queue.cc:260] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58acb114e3c436c964e5e77fa807662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } }
I20260812 03:51:51.662014 9010 raft_consensus.cc:388] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:51.662045 9010 raft_consensus.cc:482] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:51.662094 9010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:51.662734 9010 raft_consensus.cc:504] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58acb114e3c436c964e5e77fa807662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } }
I20260812 03:51:51.662827 9010 leader_election.cc:302] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a58acb114e3c436c964e5e77fa807662; no voters:
I20260812 03:51:51.662984 9010 leader_election.cc:288] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:51.663046 9021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:51.663189 9021 raft_consensus.cc:686] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [term 1 LEADER]: Becoming Leader. State: Replica: a58acb114e3c436c964e5e77fa807662, State: Running, Role: LEADER
I20260812 03:51:51.663196 9010 sys_catalog.cc:564] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:51.663328 9021 consensus_queue.cc:237] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58acb114e3c436c964e5e77fa807662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } }
I20260812 03:51:51.663645 9021 sys_catalog.cc:455] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a58acb114e3c436c964e5e77fa807662. Latest consensus state: current_term: 1 leader_uuid: "a58acb114e3c436c964e5e77fa807662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58acb114e3c436c964e5e77fa807662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } } }
I20260812 03:51:51.663739 9021 sys_catalog.cc:458] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:51.663669 9022 sys_catalog.cc:455] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a58acb114e3c436c964e5e77fa807662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a58acb114e3c436c964e5e77fa807662" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46101 } } }
I20260812 03:51:51.663861 9022 sys_catalog.cc:458] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662 [sys.catalog]: This master's current role is: LEADER
W20260812 03:51:51.665056 9051 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:51:51.665110 9051 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:51:51.665182 9033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:51.665367 9033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:51.666783 9033 catalog_manager.cc:1347] Generated new cluster ID: d36a728001284b25bde4f7040fa826fa
I20260812 03:51:51.666796 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:51.660366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:43491 (local address 127.0.0.1:46101)
0812 03:51:51.660547 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:51.660555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:51.660585 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:51:51.661552 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:51.661560 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:51.661621 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:51.661764 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:51.661979 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:51.663335 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:51.664687 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:51.665302 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:51.665711 (+ 409us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:51.666054 (+ 343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:51.666058 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:51.666058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:51.666251 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:51.666288 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:51.666301 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:51.666429 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:51.666562 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 03:51:51.666588 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260812 03:51:51.666838 9033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:51.667644 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:51.672127 9016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:51.668986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:44581 (local address 127.0.0.1:46101)
0812 03:51:51.669014 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:51.669016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:51.669075 (+ 59us) server_negotiation.cc:407] Connection header received
0812 03:51:51.669134 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:51.669134 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:51.669142 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:51.669199 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:51.669321 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:51.669935 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:51.670847 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:51.671158 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:51.671264 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:51.671871 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:51.671872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:51.671873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:51.671877 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:51.671891 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:51.671895 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:51.671962 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:51.672048 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 03:51:51.672057 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 03:51:51.675956 9033 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:51.676438 9033 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:51.687997 9033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a58acb114e3c436c964e5e77fa807662: Generated new TSK 0
I20260812 03:51:51.688153 9033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:51.690812 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 8887
W20260812 03:51:51.705649 8856 connection.cc:535] client connection to 127.0.0.1:46101 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46101): Connection reset by peer (error 104)
2026-08-12T03:51:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:51 dist-test-slave-39l8 krb5kdc[9071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:51 dist-test-slave-39l8 krb5kdc[9071](info): set up 2 sockets
Aug 12 03:51:51 dist-test-slave-39l8 krb5kdc[9071](info): commencing operation
krb5kdc: starting...
W20260812 03:51:52.856909 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:52 dist-test-slave-39l8 krb5kdc[9071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:52.963119 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:53.042263 9090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:53.042479 9090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:53.042516 9090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:53.043594 9090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:53.043661 9090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:53.043686 9090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:53.043712 9090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:53.043737 9090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:53.045082 9090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:53.045336 9090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:53.045594 9090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:53.050302 9097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:53.050333 9099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:53.050484 9090 server_base.cc:1034] running on GCE node
W20260812 03:51:53.050311 9102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:53.050881 9090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:53.051116 9090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:53.052273 9090 hybrid_clock.cc:648] HybridClock initialized: now 1786506713052257 us; error 44 us; skew 500 ppm
Aug 12 03:51:53 dist-test-slave-39l8 krb5kdc[9071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506713, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:53.058966 9090 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:53.059454 9090 webserver.cc:460] Webserver started at http://127.0.0.1:40795/ using document root <none> and password file <none>
I20260812 03:51:53.059654 9090 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:53.059698 9090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:53.059785 9090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:53.060699 9090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed"
format_stamp: "Formatted at 2026-08-12 03:51:53 on dist-test-slave-39l8"
server_key: "4a33bae95348a8dfa65f776c3f98cd14"
server_key_iv: "9e0e8e95b51c15db7dcbcfd5ebf9c15e"
server_key_version: "encryptionkey@0"
I20260812 03:51:53.060995 9090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed"
format_stamp: "Formatted at 2026-08-12 03:51:53 on dist-test-slave-39l8"
server_key: "4a33bae95348a8dfa65f776c3f98cd14"
server_key_iv: "9e0e8e95b51c15db7dcbcfd5ebf9c15e"
server_key_version: "encryptionkey@0"
I20260812 03:51:53.062233 9090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:51:53.063009 9114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:53.063189 9090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:53.063300 9090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed"
format_stamp: "Formatted at 2026-08-12 03:51:53 on dist-test-slave-39l8"
server_key: "4a33bae95348a8dfa65f776c3f98cd14"
server_key_iv: "9e0e8e95b51c15db7dcbcfd5ebf9c15e"
server_key_version: "encryptionkey@0"
I20260812 03:51:53.063377 9090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:53.066435 9090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:53.068112 9090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:53.068260 9090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:53.072359 9090 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43851
I20260812 03:51:53.072391 9222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43851 every 8 connection(s)
I20260812 03:51:53.072703 9090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:53.073664 9223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:53.075831 9223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed: Bootstrap starting.
I20260812 03:51:53.076382 9223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:53.076697 9223 log.cc:824] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:53.077328 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9090
I20260812 03:51:53.077378 9223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed: No bootstrap required, opened a new log
I20260812 03:51:53.077464 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:53.077649 7698 external_mini_cluster.cc:1428] Setting key 601990c3796282f58c755d4615b2e73e
I20260812 03:51:53.079926 9223 raft_consensus.cc:348] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43851 } }
I20260812 03:51:53.080089 9223 raft_consensus.cc:374] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:53.080147 9223 raft_consensus.cc:729] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bebedbe1d0c74cf8aa3f095c61dac3ed, State: Initialized, Role: FOLLOWER
I20260812 03:51:53.080325 9223 consensus_queue.cc:260] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43851 } }
I20260812 03:51:53.080410 9223 raft_consensus.cc:388] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:53.080432 9223 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:53.080478 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:53.078130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:39075 (local address 127.0.0.1:43851)
0812 03:51:53.078309 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:53.078318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:53.078350 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:51:53.079012 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:53.079020 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:53.079050 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:53.079183 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:53.079801 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:53.079805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:53.079805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:53.079809 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:53.079851 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:53.079864 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:53.079940 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:53.079989 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 03:51:53.080028 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260812 03:51:53.080495 9223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:53.081157 9223 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43851 } }
I20260812 03:51:53.081269 9223 leader_election.cc:302] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bebedbe1d0c74cf8aa3f095c61dac3ed; no voters:
I20260812 03:51:53.081343 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:53.081453 9223 leader_election.cc:288] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:53.081489 9228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:53.081661 9223 sys_catalog.cc:564] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:53.081625 9228 raft_consensus.cc:686] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [term 1 LEADER]: Becoming Leader. State: Replica: bebedbe1d0c74cf8aa3f095c61dac3ed, State: Running, Role: LEADER
I20260812 03:51:53.081776 9228 consensus_queue.cc:237] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43851 } }
I20260812 03:51:53.082075 9231 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43851 } } }
I20260812 03:51:53.082163 9231 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:53.082056 9228 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader bebedbe1d0c74cf8aa3f095c61dac3ed. Latest consensus state: current_term: 1 leader_uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebedbe1d0c74cf8aa3f095c61dac3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43851 } } }
I20260812 03:51:53.082336 9228 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:53.083376 9241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:53.083981 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:53.082923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:42939 (local address 127.0.0.1:43851)
0812 03:51:53.082949 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:53.082951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:53.083088 (+ 137us) server_negotiation.cc:407] Connection header received
0812 03:51:53.083199 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:53.083200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:53.083204 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:53.083260 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:53.083791 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:53.083792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:53.083793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:53.083797 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:53.083809 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:53.083813 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:53.083859 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:53.083912 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 03:51:53.083924 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260812 03:51:53.084255 9241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:53.085630 9241 catalog_manager.cc:1347] Generated new cluster ID: b1328048932d44319f113c4088a93512
I20260812 03:51:53.085690 9241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:53.127400 9241 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:53.127579 9241 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:53.130020 9241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bebedbe1d0c74cf8aa3f095c61dac3ed: Generated new TSK 0
I20260812 03:51:53.130169 9241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:53.137823 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9090
2026-08-12T03:51:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:53 dist-test-slave-39l8 krb5kdc[9267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:53 dist-test-slave-39l8 krb5kdc[9267](info): set up 2 sockets
Aug 12 03:51:53 dist-test-slave-39l8 krb5kdc[9267](info): commencing operation
krb5kdc: starting...
W20260812 03:51:54.300534 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:54 dist-test-slave-39l8 krb5kdc[9267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:54.408758 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:54.485762 9294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:54.485924 9294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:54.485944 9294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:54.486961 9294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:54.486999 9294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:54.487011 9294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:54.487022 9294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:54.487071 9294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:54.488320 9294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:54.488517 9294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:54.488706 9294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:54.493005 9305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:54.493047 9304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:54.493043 9307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:54.493443 9294 server_base.cc:1034] running on GCE node
I20260812 03:51:54.493686 9294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:54.493934 9294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:54.495088 9294 hybrid_clock.cc:648] HybridClock initialized: now 1786506714495076 us; error 25 us; skew 500 ppm
Aug 12 03:51:54 dist-test-slave-39l8 krb5kdc[9267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506714, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:54.497217 9294 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:54.497654 9294 webserver.cc:460] Webserver started at http://127.0.0.1:38593/ using document root <none> and password file <none>
I20260812 03:51:54.497856 9294 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:54.497917 9294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:54.498059 9294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:54.499123 9294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "863b230cfaed4cdba610ed90e011833d"
format_stamp: "Formatted at 2026-08-12 03:51:54 on dist-test-slave-39l8"
server_key: "d33299bfc91634ef24ecebfab080dca9"
server_key_iv: "5eee83a702a4b5f15038916169e83afa"
server_key_version: "encryptionkey@0"
I20260812 03:51:54.499471 9294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "863b230cfaed4cdba610ed90e011833d"
format_stamp: "Formatted at 2026-08-12 03:51:54 on dist-test-slave-39l8"
server_key: "d33299bfc91634ef24ecebfab080dca9"
server_key_iv: "5eee83a702a4b5f15038916169e83afa"
server_key_version: "encryptionkey@0"
I20260812 03:51:54.500707 9294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:51:54.501415 9318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:54.501626 9294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:54.501735 9294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "863b230cfaed4cdba610ed90e011833d"
format_stamp: "Formatted at 2026-08-12 03:51:54 on dist-test-slave-39l8"
server_key: "d33299bfc91634ef24ecebfab080dca9"
server_key_iv: "5eee83a702a4b5f15038916169e83afa"
server_key_version: "encryptionkey@0"
I20260812 03:51:54.501811 9294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:54.513139 9294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:54.514789 9294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:54.514945 9294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:54.518749 9294 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44067
I20260812 03:51:54.518865 9399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44067 every 8 connection(s)
I20260812 03:51:54.519090 9294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:54.519935 9400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:54.521884 9400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d: Bootstrap starting.
I20260812 03:51:54.522475 9400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:54.522800 9400 log.cc:824] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:54.523061 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9294
I20260812 03:51:54.523176 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:54.523372 7698 external_mini_cluster.cc:1428] Setting key f918b395e33c1ec50ec6c1d09aaaf683
I20260812 03:51:54.523604 9400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d: No bootstrap required, opened a new log
I20260812 03:51:54.525800 9400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863b230cfaed4cdba610ed90e011833d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44067 } }
I20260812 03:51:54.525955 9400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:54.525991 9400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863b230cfaed4cdba610ed90e011833d, State: Initialized, Role: FOLLOWER
I20260812 03:51:54.526118 9400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863b230cfaed4cdba610ed90e011833d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44067 } }
I20260812 03:51:54.526196 9400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:54.526227 9400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:54.526265 9400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:54.526865 9400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863b230cfaed4cdba610ed90e011833d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44067 } }
I20260812 03:51:54.526959 9400 leader_election.cc:302] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863b230cfaed4cdba610ed90e011833d; no voters:
I20260812 03:51:54.527084 9400 leader_election.cc:288] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:54.527158 9408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:54.527285 9400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:54.527338 9408 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [term 1 LEADER]: Becoming Leader. State: Replica: 863b230cfaed4cdba610ed90e011833d, State: Running, Role: LEADER
I20260812 03:51:54.527443 9408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863b230cfaed4cdba610ed90e011833d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44067 } }
I20260812 03:51:54.527846 9411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "863b230cfaed4cdba610ed90e011833d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863b230cfaed4cdba610ed90e011833d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44067 } } }
I20260812 03:51:54.527922 9411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:54.528090 9412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863b230cfaed4cdba610ed90e011833d. Latest consensus state: current_term: 1 leader_uuid: "863b230cfaed4cdba610ed90e011833d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863b230cfaed4cdba610ed90e011833d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44067 } } }
I20260812 03:51:54.528138 9412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:54.528496 9424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:54.529116 9424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:54.530495 9424 catalog_manager.cc:1347] Generated new cluster ID: 333d3bf065414ca8bf317cb8da052fe6
I20260812 03:51:54.530547 9424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:51:54 dist-test-slave-39l8 krb5kdc[9267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:51:54.535002 9406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:54.523959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:45625 (local address 127.0.0.1:44067)
0812 03:51:54.524207 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:54.524216 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:54.524251 (+ 35us) server_negotiation.cc:407] Connection header received
0812 03:51:54.525037 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:54.525046 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:54.525106 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:54.525248 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:54.525824 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:54.527285 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:54.528385 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:54.528985 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:54.529081 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:54.531588 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:54.531592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:54.531592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:54.531603 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:54.533445 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:54.533849 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:54.533853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:54.533853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:54.533938 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:54.534175 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:54.534175 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:54.534176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:54.534423 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:54.534556 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:54.534731 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 03:51:54.534816 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":147,"threads_started":1}
I20260812 03:51:54.535687 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:54.543525 9406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:54.536978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:49993 (local address 127.0.0.1:44067)
0812 03:51:54.537007 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:54.537008 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:54.537193 (+ 185us) server_negotiation.cc:407] Connection header received
0812 03:51:54.537241 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:54.537242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:54.537250 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:54.537295 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:54.537835 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:54.538485 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:54.539392 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:54.539661 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:54.539714 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:54.540935 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:54.540936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:54.540937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:54.540943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:54.542337 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:54.542770 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:54.542771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:54.542771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:54.542837 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:54.543069 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:54.543069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:54.543070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:54.543176 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:54.543255 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:54.543400 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:51:54.543454 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 03:51:54.545516 9424 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:54.546177 9424 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:54.557725 9424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863b230cfaed4cdba610ed90e011833d: Generated new TSK 0
I20260812 03:51:54.557883 9424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:54.559638 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9294
W20260812 03:51:54.569430 9265 connection.cc:535] client connection to 127.0.0.1:44067 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44067): Connection reset by peer (error 104)
2026-08-12T03:51:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:54 dist-test-slave-39l8 krb5kdc[9455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:54 dist-test-slave-39l8 krb5kdc[9455](info): set up 2 sockets
Aug 12 03:51:54 dist-test-slave-39l8 krb5kdc[9455](info): commencing operation
krb5kdc: starting...
W20260812 03:51:55.710024 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:55 dist-test-slave-39l8 krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:55.819551 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34819
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:55.896870 9472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:55.897073 9472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:55.897109 9472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:55.898139 9472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:55.898202 9472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:55.898227 9472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:55.898257 9472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:55.898278 9472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:55.899739 9472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34819
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:55.899992 9472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:55.900199 9472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:55.904699 9481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:55.904686 9478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:55.904704 9479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:55.904974 9472 server_base.cc:1034] running on GCE node
I20260812 03:51:55.905144 9472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:55.905395 9472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:55.906553 9472 hybrid_clock.cc:648] HybridClock initialized: now 1786506715906534 us; error 28 us; skew 500 ppm
Aug 12 03:51:55 dist-test-slave-39l8 krb5kdc[9455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506715, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:55.908933 9472 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:55.909358 9472 webserver.cc:460] Webserver started at http://127.0.0.1:45043/ using document root <none> and password file <none>
I20260812 03:51:55.909555 9472 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:55.909593 9472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:55.909685 9472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:55.910624 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "d3fc828ad7564930a5b59482df0fa060"
format_stamp: "Formatted at 2026-08-12 03:51:55 on dist-test-slave-39l8"
server_key: "eac4aea6013e07952c578eeb6c2fadc4"
server_key_iv: "3f1f9725f850824dc7ded04c7492363a"
server_key_version: "encryptionkey@0"
I20260812 03:51:55.911015 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "d3fc828ad7564930a5b59482df0fa060"
format_stamp: "Formatted at 2026-08-12 03:51:55 on dist-test-slave-39l8"
server_key: "eac4aea6013e07952c578eeb6c2fadc4"
server_key_iv: "3f1f9725f850824dc7ded04c7492363a"
server_key_version: "encryptionkey@0"
I20260812 03:51:55.912267 9472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:51:55.912953 9492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:55.913123 9472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:55.913213 9472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "d3fc828ad7564930a5b59482df0fa060"
format_stamp: "Formatted at 2026-08-12 03:51:55 on dist-test-slave-39l8"
server_key: "eac4aea6013e07952c578eeb6c2fadc4"
server_key_iv: "3f1f9725f850824dc7ded04c7492363a"
server_key_version: "encryptionkey@0"
I20260812 03:51:55.913267 9472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:55.925765 9472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:55.927944 9472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:55.928046 9472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:55.931810 9472 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34819
I20260812 03:51:55.931864 9586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34819 every 8 connection(s)
I20260812 03:51:55.932205 9472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:55.933106 9587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:55.934326 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9472
I20260812 03:51:55.934437 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:55.934643 7698 external_mini_cluster.cc:1428] Setting key c0ee848c2b142dbf067da4c1460587ee
I20260812 03:51:55.935801 9587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060: Bootstrap starting.
I20260812 03:51:55.936528 9587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:55.936818 9587 log.cc:824] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:55.937522 9587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060: No bootstrap required, opened a new log
I20260812 03:51:55.939677 9587 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fc828ad7564930a5b59482df0fa060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34819 } }
I20260812 03:51:55.939850 9587 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:55.939951 9587 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3fc828ad7564930a5b59482df0fa060, State: Initialized, Role: FOLLOWER
I20260812 03:51:55.940145 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:55.935249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:35757 (local address 127.0.0.1:34819)
0812 03:51:55.935415 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:55.935423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:55.935445 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:51:55.935806 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:55.935811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:55.935877 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:55.936036 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:55.936315 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:55.937356 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:55.938350 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:55.938781 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:55.938854 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:55.939498 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:55.939502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:55.939502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:55.939646 (+ 144us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:55.939672 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:55.939681 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:55.939741 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:55.939925 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 03:51:55.939957 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260812 03:51:55.940114 9587 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fc828ad7564930a5b59482df0fa060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34819 } }
I20260812 03:51:55.940228 9587 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:55.940265 9587 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:55.940315 9587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:55.940924 9587 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fc828ad7564930a5b59482df0fa060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34819 } }
I20260812 03:51:55.941011 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:55.941028 9587 leader_election.cc:302] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3fc828ad7564930a5b59482df0fa060; no voters:
I20260812 03:51:55.941197 9587 leader_election.cc:288] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:55.941233 9595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:55.941427 9595 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [term 1 LEADER]: Becoming Leader. State: Replica: d3fc828ad7564930a5b59482df0fa060, State: Running, Role: LEADER
I20260812 03:51:55.941432 9587 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:55.942343 9595 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fc828ad7564930a5b59482df0fa060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34819 } }
I20260812 03:51:55.943042 9598 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3fc828ad7564930a5b59482df0fa060. Latest consensus state: current_term: 1 leader_uuid: "d3fc828ad7564930a5b59482df0fa060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fc828ad7564930a5b59482df0fa060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34819 } } }
I20260812 03:51:55.943185 9600 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3fc828ad7564930a5b59482df0fa060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3fc828ad7564930a5b59482df0fa060" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34819 } } }
W20260812 03:51:55.943390 9626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:51:55.943873 9626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:51:55.943862 9600 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:55.943852 9598 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:55.944154 9630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:55.944319 9630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:55.945875 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:55.942447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:38143 (local address 127.0.0.1:34819)
0812 03:51:55.942534 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:55.942536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:55.942587 (+ 51us) server_negotiation.cc:407] Connection header received
0812 03:51:55.942669 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:55.942670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:55.942679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:55.942754 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:55.942913 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:55.943549 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:55.944535 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:55.944803 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:55.944867 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:55.945624 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:55.945625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:55.945626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:55.945630 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:55.945644 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:55.945648 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:55.945721 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:55.945776 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:51:55.945787 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:51:55.946439 9630 catalog_manager.cc:1347] Generated new cluster ID: 16828ade25f449768504265b3df66a76
I20260812 03:51:55.946496 9630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:55.955785 9630 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:55.956403 9630 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:55.975884 9630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3fc828ad7564930a5b59482df0fa060: Generated new TSK 0
I20260812 03:51:55.976065 9630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:55.984004 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9472
W20260812 03:51:56.000597 9452 connection.cc:535] client connection to 127.0.0.1:34819 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34819): Connection reset by peer (error 104)
2026-08-12T03:51:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:56 dist-test-slave-39l8 krb5kdc[9640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:56 dist-test-slave-39l8 krb5kdc[9640](info): set up 2 sockets
Aug 12 03:51:56 dist-test-slave-39l8 krb5kdc[9640](info): commencing operation
krb5kdc: starting...
W20260812 03:51:57.153455 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:57 dist-test-slave-39l8 krb5kdc[9640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:51:57.267303 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:57.346586 9659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:57.346786 9659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:57.346807 9659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:57.347770 9659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:57.347802 9659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:57.347815 9659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:57.347826 9659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:57.347836 9659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:57.349103 9659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41137
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:57.349274 9659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:57.349422 9659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:57.354060 9677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:57.354054 9678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:57.354058 9680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:57.354310 9659 server_base.cc:1034] running on GCE node
I20260812 03:51:57.354538 9659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:57.354885 9659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:57.356046 9659 hybrid_clock.cc:648] HybridClock initialized: now 1786506717356025 us; error 35 us; skew 500 ppm
Aug 12 03:51:57 dist-test-slave-39l8 krb5kdc[9640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506717, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:57.358332 9659 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:51:57.358795 9659 webserver.cc:460] Webserver started at http://127.0.0.1:46789/ using document root <none> and password file <none>
I20260812 03:51:57.359010 9659 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:57.359074 9659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:57.359202 9659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:57.360152 9659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "4bf04f023f9b4cb0913464e8526425a7"
format_stamp: "Formatted at 2026-08-12 03:51:57 on dist-test-slave-39l8"
server_key: "41146e8174cfdb03eaed448b84f1c891"
server_key_iv: "6a576cb8e8f83a704e4656cd221cf8c5"
server_key_version: "encryptionkey@0"
I20260812 03:51:57.360502 9659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "4bf04f023f9b4cb0913464e8526425a7"
format_stamp: "Formatted at 2026-08-12 03:51:57 on dist-test-slave-39l8"
server_key: "41146e8174cfdb03eaed448b84f1c891"
server_key_iv: "6a576cb8e8f83a704e4656cd221cf8c5"
server_key_version: "encryptionkey@0"
I20260812 03:51:57.361826 9659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:51:57.362514 9689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:57.362690 9659 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:57.362834 9659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "4bf04f023f9b4cb0913464e8526425a7"
format_stamp: "Formatted at 2026-08-12 03:51:57 on dist-test-slave-39l8"
server_key: "41146e8174cfdb03eaed448b84f1c891"
server_key_iv: "6a576cb8e8f83a704e4656cd221cf8c5"
server_key_version: "encryptionkey@0"
I20260812 03:51:57.362932 9659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:57.366024 9659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:57.367792 9659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:57.367946 9659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:57.371948 9659 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41137
I20260812 03:51:57.371979 9773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41137 every 8 connection(s)
I20260812 03:51:57.372313 9659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:57.373142 9774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:57.375353 9774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7: Bootstrap starting.
I20260812 03:51:57.375932 9774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:57.376216 9774 log.cc:824] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:57.376870 9774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7: No bootstrap required, opened a new log
I20260812 03:51:57.378405 9774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bf04f023f9b4cb0913464e8526425a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41137 } }
I20260812 03:51:57.378535 9774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:57.378592 9774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bf04f023f9b4cb0913464e8526425a7, State: Initialized, Role: FOLLOWER
I20260812 03:51:57.378780 9774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bf04f023f9b4cb0913464e8526425a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41137 } }
I20260812 03:51:57.378851 9774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:57.378891 9774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:57.378940 9774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:57.379449 9774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bf04f023f9b4cb0913464e8526425a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41137 } }
I20260812 03:51:57.379561 9774 leader_election.cc:302] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bf04f023f9b4cb0913464e8526425a7; no voters:
I20260812 03:51:57.379741 9774 leader_election.cc:288] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:57.379765 9777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:51:57.379935 9777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [term 1 LEADER]: Becoming Leader. State: Replica: 4bf04f023f9b4cb0913464e8526425a7, State: Running, Role: LEADER
I20260812 03:51:57.380013 9774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:57.380065 9777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bf04f023f9b4cb0913464e8526425a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41137 } }
I20260812 03:51:57.380420 9779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bf04f023f9b4cb0913464e8526425a7. Latest consensus state: current_term: 1 leader_uuid: "4bf04f023f9b4cb0913464e8526425a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bf04f023f9b4cb0913464e8526425a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41137 } } }
I20260812 03:51:57.380543 9778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bf04f023f9b4cb0913464e8526425a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bf04f023f9b4cb0913464e8526425a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41137 } } }
I20260812 03:51:57.380546 9779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:57.380641 9778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:57.381078 9797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:57.381502 9797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:57.381870 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9659
I20260812 03:51:57.381999 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:57.382144 7698 external_mini_cluster.cc:1428] Setting key 6b3e44ab5ee5f129c0c76ea1aedbe2bb
I20260812 03:51:57.383788 9797 catalog_manager.cc:1347] Generated new cluster ID: 83f2cd14b45c4f6b8bc7782bf7c0734d
I20260812 03:51:57.383844 9797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:57.384325 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:57.382548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:47983 (local address 127.0.0.1:41137)
0812 03:51:57.382764 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:57.382773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:57.382866 (+ 93us) server_negotiation.cc:407] Connection header received
0812 03:51:57.383034 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:57.383041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:57.383070 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:57.383218 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:57.383844 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:57.383847 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:57.383848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:57.383853 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:57.383898 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:57.383912 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:57.383967 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:57.384121 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 03:51:57.384160 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260812 03:51:57.385149 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:57.387827 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:57.386522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:36095 (local address 127.0.0.1:41137)
0812 03:51:57.386552 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:57.386554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:57.386955 (+ 401us) server_negotiation.cc:407] Connection header received
0812 03:51:57.387034 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:57.387035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:57.387039 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:57.387062 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:57.387632 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:57.387633 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:57.387634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:51:57.387637 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:57.387648 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:51:57.387652 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:57.387676 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:57.387761 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 03:51:57.387773 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 03:51:57.396139 9797 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:57.396294 9797 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:57.402877 9797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bf04f023f9b4cb0913464e8526425a7: Generated new TSK 0
I20260812 03:51:57.403050 9797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:57.404663 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9659
2026-08-12T03:51:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:57 dist-test-slave-39l8 krb5kdc[9828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:57 dist-test-slave-39l8 krb5kdc[9828](info): set up 2 sockets
Aug 12 03:51:57 dist-test-slave-39l8 krb5kdc[9828](info): commencing operation
krb5kdc: starting...
W20260812 03:51:58.578897 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:51:58 dist-test-slave-39l8 krb5kdc[9828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:51:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:51:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:51:58.686812 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34157
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:34157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:51:58.764272 9843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:51:58.764479 9843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:51:58.764514 9843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:51:58.765599 9843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:51:58.765671 9843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:51:58.765697 9843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:51:58.765727 9843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:51:58.765750 9843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:51:58.767177 9843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:34157
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:34157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:51:58.767427 9843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:51:58.767628 9843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:51:58.772188 9856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:58.772214 9843 server_base.cc:1034] running on GCE node
W20260812 03:51:58.772181 9851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:51:58.772179 9850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:51:58.772652 9843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:51:58.772962 9843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:51:58.774099 9843 hybrid_clock.cc:648] HybridClock initialized: now 1786506718774096 us; error 35 us; skew 500 ppm
Aug 12 03:51:58 dist-test-slave-39l8 krb5kdc[9828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506718, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:51:58.780237 9843 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:51:58.780642 9843 webserver.cc:460] Webserver started at http://127.7.132.190:46547/ using document root <none> and password file <none>
I20260812 03:51:58.780862 9843 fs_manager.cc:359] Metadata directory not provided
I20260812 03:51:58.780953 9843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:51:58.781090 9843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:51:58.782064 9843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "412be1e25e404f3cad003dd2cf0e96c2"
format_stamp: "Formatted at 2026-08-12 03:51:58 on dist-test-slave-39l8"
server_key: "05164a305320cf3bbc2169c16693c415"
server_key_iv: "4bd76d2da769c6f07d39ab9f0865a35c"
server_key_version: "encryptionkey@0"
I20260812 03:51:58.782429 9843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "412be1e25e404f3cad003dd2cf0e96c2"
format_stamp: "Formatted at 2026-08-12 03:51:58 on dist-test-slave-39l8"
server_key: "05164a305320cf3bbc2169c16693c415"
server_key_iv: "4bd76d2da769c6f07d39ab9f0865a35c"
server_key_version: "encryptionkey@0"
I20260812 03:51:58.783780 9843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:51:58.784433 9869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:51:58.784602 9843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:51:58.784698 9843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "412be1e25e404f3cad003dd2cf0e96c2"
format_stamp: "Formatted at 2026-08-12 03:51:58 on dist-test-slave-39l8"
server_key: "05164a305320cf3bbc2169c16693c415"
server_key_iv: "4bd76d2da769c6f07d39ab9f0865a35c"
server_key_version: "encryptionkey@0"
I20260812 03:51:58.784801 9843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:51:58.793658 9843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:51:58.795249 9843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:51:58.795395 9843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:51:58.799146 9843 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:34157
I20260812 03:51:58.799175 9959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:34157 every 8 connection(s)
I20260812 03:51:58.799477 9843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:51:58.800297 9960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:51:58.801373 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 9843
I20260812 03:51:58.801498 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:51:58.801688 7698 external_mini_cluster.cc:1428] Setting key 2f3c601a790ae511960b43eb4cb9ee3f
I20260812 03:51:58.802969 9960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2: Bootstrap starting.
I20260812 03:51:58.803695 9960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2: Neither blocks nor log segments found. Creating new log.
I20260812 03:51:58.804009 9960 log.cc:824] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:51:58.804730 9960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2: No bootstrap required, opened a new log
I20260812 03:51:58.806692 9960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412be1e25e404f3cad003dd2cf0e96c2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34157 } }
I20260812 03:51:58.806890 9960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:51:58.806938 9960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412be1e25e404f3cad003dd2cf0e96c2, State: Initialized, Role: FOLLOWER
I20260812 03:51:58.807054 9960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412be1e25e404f3cad003dd2cf0e96c2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34157 } }
I20260812 03:51:58.807132 9960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:51:58.807176 9960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:51:58.807221 9960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:51:58.807772 9960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412be1e25e404f3cad003dd2cf0e96c2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34157 } }
I20260812 03:51:58.807881 9960 leader_election.cc:302] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 412be1e25e404f3cad003dd2cf0e96c2; no voters:
I20260812 03:51:58.808048 9960 leader_election.cc:288] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:51:58.808108 9970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 1 FOLLOWER]: Leader election won for term 1
Aug 12 03:51:58 dist-test-slave-39l8 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:51:58.808315 9960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:51:58.808298 9970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [term 1 LEADER]: Becoming Leader. State: Replica: 412be1e25e404f3cad003dd2cf0e96c2, State: Running, Role: LEADER
I20260812 03:51:58.808414 9970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412be1e25e404f3cad003dd2cf0e96c2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34157 } }
I20260812 03:51:58.808851 9972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 412be1e25e404f3cad003dd2cf0e96c2. Latest consensus state: current_term: 1 leader_uuid: "412be1e25e404f3cad003dd2cf0e96c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412be1e25e404f3cad003dd2cf0e96c2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34157 } } }
I20260812 03:51:58.808925 9972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:58.809130 9979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:51:58.809252 9971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "412be1e25e404f3cad003dd2cf0e96c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412be1e25e404f3cad003dd2cf0e96c2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 34157 } } }
I20260812 03:51:58.809306 9971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2 [sys.catalog]: This master's current role is: LEADER
I20260812 03:51:58.809620 9979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:51:58.811246 9979 catalog_manager.cc:1347] Generated new cluster ID: ce8ca5af692542e3b9dcb1ae07bb4afc
I20260812 03:51:58.811292 9979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:51:58.812642 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:58.802192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55119 (local address 127.7.132.190:34157)
0812 03:51:58.802372 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:58.802379 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:58.802415 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:51:58.802907 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:58.802914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:58.802959 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:58.803135 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:58.803894 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:58.804798 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:58.805577 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:58.805978 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:58.806099 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:58.808787 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:58.808792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:58.808792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:58.808805 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:58.810907 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:58.811380 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:58.811383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:58.811384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:58.811481 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:58.811725 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:58.811726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:58.811726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:58.812044 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:58.812210 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:58.812343 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 03:51:58.812446 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260812 03:51:58.813619 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:51:58.822146 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:51:58.815109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56071 (local address 127.7.132.190:34157)
0812 03:51:58.815142 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:51:58.815143 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:51:58.815195 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:51:58.815248 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:51:58.815249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:51:58.815257 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:51:58.815322 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:51:58.815855 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:58.816854 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:58.817721 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:51:58.818008 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:51:58.818078 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:51:58.819322 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:51:58.819323 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:51:58.819323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:51:58.819330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:51:58.820960 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:58.821336 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:58.821337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:58.821337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:58.821396 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:51:58.821608 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:51:58.821609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:51:58.821609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:51:58.821748 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:51:58.821847 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:51:58.821991 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:51:58.822057 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:51:58.829355 9979 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:51:58.829794 9979 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:51:58.832742 9979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 412be1e25e404f3cad003dd2cf0e96c2: Generated new TSK 0
I20260812 03:51:58.832916 9979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:51:58.838074 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 9843
W20260812 03:51:58.854868 9825 connection.cc:535] client connection to 127.7.132.190:34157 recv error: Network error: failed to read from TLS socket (remote: 127.7.132.190:34157): Connection reset by peer (error 104)
2026-08-12T03:51:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1436 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:51:58 dist-test-slave-39l8 krb5kdc[10005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:51:58 dist-test-slave-39l8 krb5kdc[10005](info): set up 2 sockets
Aug 12 03:51:58 dist-test-slave-39l8 krb5kdc[10005](info): commencing operation
krb5kdc: starting...
W20260812 03:52:00.031639 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:52:00 dist-test-slave-39l8 krb5kdc[10005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:52:00.146935 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38705
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:00.225692 10022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:00.225914 10022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:00.225948 10022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:00.227079 10022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:00.227144 10022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:00.227165 10022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:00.227185 10022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:00.227214 10022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:00.228667 10022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38705
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:00.228932 10022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:00.229154 10022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:00.234068 10035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:00.234059 10031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:00.234081 10033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:00.234303 10022 server_base.cc:1034] running on GCE node
I20260812 03:52:00.234546 10022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:00.234859 10022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:00.236012 10022 hybrid_clock.cc:648] HybridClock initialized: now 1786506720235989 us; error 40 us; skew 500 ppm
Aug 12 03:52:00 dist-test-slave-39l8 krb5kdc[10005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506720, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:00.238358 10022 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:52:00.238876 10022 webserver.cc:460] Webserver started at http://127.7.132.190:33805/ using document root <none> and password file <none>
I20260812 03:52:00.239120 10022 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:00.239164 10022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:00.239317 10022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:00.240376 10022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "207941f69ce443e2b0aefb5e5e3957b3"
format_stamp: "Formatted at 2026-08-12 03:52:00 on dist-test-slave-39l8"
server_key: "4693fb1dfb3e6ed5021ea951628a0607"
server_key_iv: "1306f0ccef9e05ca460c13c0dc976870"
server_key_version: "encryptionkey@0"
I20260812 03:52:00.240723 10022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "207941f69ce443e2b0aefb5e5e3957b3"
format_stamp: "Formatted at 2026-08-12 03:52:00 on dist-test-slave-39l8"
server_key: "4693fb1dfb3e6ed5021ea951628a0607"
server_key_iv: "1306f0ccef9e05ca460c13c0dc976870"
server_key_version: "encryptionkey@0"
I20260812 03:52:00.241982 10022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:00.242776 10049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:00.242949 10022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:00.243042 10022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "207941f69ce443e2b0aefb5e5e3957b3"
format_stamp: "Formatted at 2026-08-12 03:52:00 on dist-test-slave-39l8"
server_key: "4693fb1dfb3e6ed5021ea951628a0607"
server_key_iv: "1306f0ccef9e05ca460c13c0dc976870"
server_key_version: "encryptionkey@0"
I20260812 03:52:00.243114 10022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:00.269994 10022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:00.272722 10022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:00.272895 10022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:00.276943 10022 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38705
I20260812 03:52:00.277105 10130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38705 every 8 connection(s)
I20260812 03:52:00.277324 10022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:00.278146 10131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:00.280334 10131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3: Bootstrap starting.
I20260812 03:52:00.280829 10131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:00.281071 10131 log.cc:824] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:00.281440 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10022
I20260812 03:52:00.281556 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:52:00.281739 7698 external_mini_cluster.cc:1428] Setting key 6cb9d137d11444ff2834837b48a02c2d
I20260812 03:52:00.281777 10131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3: No bootstrap required, opened a new log
I20260812 03:52:00.284345 10131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207941f69ce443e2b0aefb5e5e3957b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38705 } }
I20260812 03:52:00.284531 10131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:00.284574 10131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 207941f69ce443e2b0aefb5e5e3957b3, State: Initialized, Role: FOLLOWER
I20260812 03:52:00.284693 10131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207941f69ce443e2b0aefb5e5e3957b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38705 } }
I20260812 03:52:00.284768 10131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:00.284795 10131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:00.284835 10131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:00.285425 10131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207941f69ce443e2b0aefb5e5e3957b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38705 } }
I20260812 03:52:00.285509 10131 leader_election.cc:302] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 207941f69ce443e2b0aefb5e5e3957b3; no voters:
I20260812 03:52:00.285705 10131 leader_election.cc:288] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:00.285751 10141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:00.285902 10141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [term 1 LEADER]: Becoming Leader. State: Replica: 207941f69ce443e2b0aefb5e5e3957b3, State: Running, Role: LEADER
I20260812 03:52:00.285961 10131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:00.286073 10141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207941f69ce443e2b0aefb5e5e3957b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38705 } }
I20260812 03:52:00.286432 10141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 207941f69ce443e2b0aefb5e5e3957b3. Latest consensus state: current_term: 1 leader_uuid: "207941f69ce443e2b0aefb5e5e3957b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207941f69ce443e2b0aefb5e5e3957b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38705 } } }
I20260812 03:52:00.286552 10141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:00.286638 10144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "207941f69ce443e2b0aefb5e5e3957b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "207941f69ce443e2b0aefb5e5e3957b3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38705 } } }
I20260812 03:52:00.286755 10144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:00.287566 10160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:00.287715 10139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:00.282256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59119 (local address 127.7.132.190:38705)
0812 03:52:00.282431 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:00.282439 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:00.282471 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:52:00.283142 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:00.283149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:00.283208 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:00.283329 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:00.283560 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:00.284309 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:00.285341 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:00.285863 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:00.285971 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:00.286807 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:00.286811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:00.286811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:00.287124 (+ 313us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:00.287162 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:00.287177 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:00.287264 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:00.287450 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:52:00.287477 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260812 03:52:00.288223 10160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:00.288708 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:00.289710 10160 catalog_manager.cc:1347] Generated new cluster ID: ea296048117a4996a0e3552b52100208
I20260812 03:52:00.289750 10160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:00.293164 10139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:00.289942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42221 (local address 127.7.132.190:38705)
0812 03:52:00.289974 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:00.289976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:00.290126 (+ 150us) server_negotiation.cc:407] Connection header received
0812 03:52:00.290199 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:00.290199 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:00.290206 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:00.290238 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:00.290393 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:00.291007 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:00.291909 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:00.292244 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:00.292278 (+ 34us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:00.292918 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:00.292919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:00.292919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:00.292923 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:00.292934 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:00.292938 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:00.292965 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:00.293087 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 03:52:00.293097 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:52:00.297995 10160 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:00.298593 10160 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:00.305583 10160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 207941f69ce443e2b0aefb5e5e3957b3: Generated new TSK 0
I20260812 03:52:00.305727 10160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:00.308604 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10022
W20260812 03:52:00.319962 10002 connection.cc:535] client connection to 127.7.132.190:38705 recv error: Network error: failed to read from TLS socket (remote: 127.7.132.190:38705): Connection reset by peer (error 104)
2026-08-12T03:52:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:00 dist-test-slave-39l8 krb5kdc[10177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:00 dist-test-slave-39l8 krb5kdc[10177](info): set up 2 sockets
Aug 12 03:52:00 dist-test-slave-39l8 krb5kdc[10177](info): commencing operation
krb5kdc: starting...
W20260812 03:52:01.495694 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:52:01 dist-test-slave-39l8 krb5kdc[10177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:52:01.607102 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38541
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:38541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:01.684674 10204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:01.684863 10204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:01.684885 10204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:01.686033 10204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:01.686074 10204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:01.686086 10204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:01.686100 10204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:01.686148 10204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:01.687503 10204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:38541
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:38541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:01.687707 10204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:01.687945 10204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:01.692548 10215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:01.692603 10214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:01.692557 10218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:01.692659 10204 server_base.cc:1034] running on GCE node
I20260812 03:52:01.692992 10204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:01.693261 10204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:01.694408 10204 hybrid_clock.cc:648] HybridClock initialized: now 1786506721694395 us; error 29 us; skew 500 ppm
Aug 12 03:52:01 dist-test-slave-39l8 krb5kdc[10177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506721, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:01.696846 10204 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:52:01.697216 10204 webserver.cc:460] Webserver started at http://127.7.132.190:41401/ using document root <none> and password file <none>
I20260812 03:52:01.697388 10204 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:01.697427 10204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:01.697512 10204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:01.698447 10204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "b76ba577723f4670a20ba9f285795bf4"
format_stamp: "Formatted at 2026-08-12 03:52:01 on dist-test-slave-39l8"
server_key: "e04a404278230cf61597245d09cf323c"
server_key_iv: "08d5906aabe44a285f1b24ca8e662f1d"
server_key_version: "encryptionkey@0"
I20260812 03:52:01.698805 10204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "b76ba577723f4670a20ba9f285795bf4"
format_stamp: "Formatted at 2026-08-12 03:52:01 on dist-test-slave-39l8"
server_key: "e04a404278230cf61597245d09cf323c"
server_key_iv: "08d5906aabe44a285f1b24ca8e662f1d"
server_key_version: "encryptionkey@0"
I20260812 03:52:01.699988 10204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:52:01.700628 10234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:01.700783 10204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:52:01.700871 10204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "b76ba577723f4670a20ba9f285795bf4"
format_stamp: "Formatted at 2026-08-12 03:52:01 on dist-test-slave-39l8"
server_key: "e04a404278230cf61597245d09cf323c"
server_key_iv: "08d5906aabe44a285f1b24ca8e662f1d"
server_key_version: "encryptionkey@0"
I20260812 03:52:01.700925 10204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:01.704058 10204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:01.705735 10204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:01.705835 10204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:01.709635 10204 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:38541
I20260812 03:52:01.709707 10318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:38541 every 8 connection(s)
I20260812 03:52:01.709990 10204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:01.711040 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:01.711458 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10204
I20260812 03:52:01.711578 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:52:01.711767 7698 external_mini_cluster.cc:1428] Setting key ca606a68520926dc3fbd0e7723e51816
I20260812 03:52:01.714164 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:01.712309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52587 (local address 127.7.132.190:38541)
0812 03:52:01.712511 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:01.712518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:01.712542 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:52:01.712954 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:01.712960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:01.712991 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:01.713102 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:01.713754 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:01.713757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:01.713757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:01.713761 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:01.713795 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:01.713803 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:01.713854 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:01.713946 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 03:52:01.713973 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260812 03:52:01.714202 10320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4: Bootstrap starting.
I20260812 03:52:01.714972 10320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:01.715111 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:01.715322 10320 log.cc:824] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:01.715996 10320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4: No bootstrap required, opened a new log
I20260812 03:52:01.717563 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:01.716468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33131 (local address 127.7.132.190:38541)
0812 03:52:01.716503 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:01.716505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:01.716636 (+ 131us) server_negotiation.cc:407] Connection header received
0812 03:52:01.716713 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:01.716714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:01.716719 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:01.716779 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:01.717363 (+ 584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:01.717364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:01.717364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:01.717368 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:01.717380 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:01.717383 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:01.717427 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:01.717479 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 03:52:01.717491 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:52:01.718186 10320 raft_consensus.cc:348] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76ba577723f4670a20ba9f285795bf4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38541 } }
I20260812 03:52:01.718313 10320 raft_consensus.cc:374] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:01.718334 10320 raft_consensus.cc:729] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b76ba577723f4670a20ba9f285795bf4, State: Initialized, Role: FOLLOWER
I20260812 03:52:01.718436 10320 consensus_queue.cc:260] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76ba577723f4670a20ba9f285795bf4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38541 } }
I20260812 03:52:01.718518 10320 raft_consensus.cc:388] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:01.718538 10320 raft_consensus.cc:482] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:01.718564 10320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:01.719081 10320 raft_consensus.cc:504] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76ba577723f4670a20ba9f285795bf4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38541 } }
I20260812 03:52:01.719187 10320 leader_election.cc:302] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b76ba577723f4670a20ba9f285795bf4; no voters:
I20260812 03:52:01.719298 10320 leader_election.cc:288] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:01.719362 10334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:01.719494 10334 raft_consensus.cc:686] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [term 1 LEADER]: Becoming Leader. State: Replica: b76ba577723f4670a20ba9f285795bf4, State: Running, Role: LEADER
I20260812 03:52:01.719508 10320 sys_catalog.cc:564] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:01.720717 10334 consensus_queue.cc:237] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76ba577723f4670a20ba9f285795bf4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38541 } }
I20260812 03:52:01.721290 10335 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b76ba577723f4670a20ba9f285795bf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76ba577723f4670a20ba9f285795bf4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38541 } } }
I20260812 03:52:01.721622 10335 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:01.721289 10337 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b76ba577723f4670a20ba9f285795bf4. Latest consensus state: current_term: 1 leader_uuid: "b76ba577723f4670a20ba9f285795bf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76ba577723f4670a20ba9f285795bf4" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 38541 } } }
I20260812 03:52:01.721673 10337 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4 [sys.catalog]: This master's current role is: LEADER
W20260812 03:52:01.721820 10359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:01.721872 10359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:01.721994 10362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:01.722198 10362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:01.723758 10362 catalog_manager.cc:1347] Generated new cluster ID: 134b0cc3648547259016ac701066efbe
I20260812 03:52:01.723798 10362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:01.731590 10362 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:01.731737 10362 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:01.735366 10362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b76ba577723f4670a20ba9f285795bf4: Generated new TSK 0
I20260812 03:52:01.735517 10362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:01.742071 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10204
2026-08-12T03:52:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:01 dist-test-slave-39l8 krb5kdc[10371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:01 dist-test-slave-39l8 krb5kdc[10371](info): set up 2 sockets
Aug 12 03:52:01 dist-test-slave-39l8 krb5kdc[10371](info): commencing operation
krb5kdc: starting...
W20260812 03:52:02.933176 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:52:03 dist-test-slave-39l8 krb5kdc[10371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:52:03.043481 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33011
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:34931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:33011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:03.122874 10392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:03.123059 10392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:03.123081 10392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:03.124307 10392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:03.124348 10392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:03.124362 10392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:03.124375 10392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:03.124423 10392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:03.125792 10392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:34931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:33011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:33011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:03.126025 10392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:03.126252 10392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:03.130617 10402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:03.130659 10404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:03.130626 10400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:03.130792 10392 server_base.cc:1034] running on GCE node
I20260812 03:52:03.130998 10392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:03.131249 10392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:03.132388 10392 hybrid_clock.cc:648] HybridClock initialized: now 1786506723132367 us; error 35 us; skew 500 ppm
Aug 12 03:52:03 dist-test-slave-39l8 krb5kdc[10371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506723, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:03.134883 10392 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:52:03.135298 10392 webserver.cc:460] Webserver started at http://127.7.132.190:39067/ using document root <none> and password file <none>
I20260812 03:52:03.135495 10392 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:03.135536 10392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:03.135668 10392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:03.136583 10392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "f6993c8e2d624b9891e1879afc71ee3f"
format_stamp: "Formatted at 2026-08-12 03:52:03 on dist-test-slave-39l8"
server_key: "7fce5ed5a95e8e96ef35bc1486d553b8"
server_key_iv: "2c391da421e2f98d6454f364a7d3bc6e"
server_key_version: "encryptionkey@0"
I20260812 03:52:03.136932 10392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "f6993c8e2d624b9891e1879afc71ee3f"
format_stamp: "Formatted at 2026-08-12 03:52:03 on dist-test-slave-39l8"
server_key: "7fce5ed5a95e8e96ef35bc1486d553b8"
server_key_iv: "2c391da421e2f98d6454f364a7d3bc6e"
server_key_version: "encryptionkey@0"
I20260812 03:52:03.138195 10392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:03.138850 10412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:03.139011 10392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:03.139081 10392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "f6993c8e2d624b9891e1879afc71ee3f"
format_stamp: "Formatted at 2026-08-12 03:52:03 on dist-test-slave-39l8"
server_key: "7fce5ed5a95e8e96ef35bc1486d553b8"
server_key_iv: "2c391da421e2f98d6454f364a7d3bc6e"
server_key_version: "encryptionkey@0"
I20260812 03:52:03.139165 10392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:03.152864 10392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:03.154984 10392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:03.155133 10392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:03.158905 10392 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:33011
I20260812 03:52:03.158948 10508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:33011 every 8 connection(s)
I20260812 03:52:03.159232 10392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:03.159983 10509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:03.162448 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f: Bootstrap starting.
I20260812 03:52:03.163082 10509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:03.163378 10509 log.cc:824] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:03.164047 10509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f: No bootstrap required, opened a new log
I20260812 03:52:03.165465 10509 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6993c8e2d624b9891e1879afc71ee3f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33011 } }
I20260812 03:52:03.165642 10509 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:03.165692 10509 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6993c8e2d624b9891e1879afc71ee3f, State: Initialized, Role: FOLLOWER
I20260812 03:52:03.165854 10509 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6993c8e2d624b9891e1879afc71ee3f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33011 } }
I20260812 03:52:03.165954 10509 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:03.165999 10509 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:03.166047 10509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:03.166909 10509 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6993c8e2d624b9891e1879afc71ee3f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33011 } }
I20260812 03:52:03.167017 10509 leader_election.cc:302] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6993c8e2d624b9891e1879afc71ee3f; no voters:
I20260812 03:52:03.167173 10509 leader_election.cc:288] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:03.167217 10515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:03.167383 10515 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [term 1 LEADER]: Becoming Leader. State: Replica: f6993c8e2d624b9891e1879afc71ee3f, State: Running, Role: LEADER
I20260812 03:52:03.167407 10509 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:03.167488 10515 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6993c8e2d624b9891e1879afc71ee3f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33011 } }
I20260812 03:52:03.167786 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10392
I20260812 03:52:03.167917 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:52:03.168146 7698 external_mini_cluster.cc:1428] Setting key 55e474ff8374a4bcc51f963eacff7992
I20260812 03:52:03.169096 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6993c8e2d624b9891e1879afc71ee3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6993c8e2d624b9891e1879afc71ee3f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33011 } } }
I20260812 03:52:03.169085 10521 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6993c8e2d624b9891e1879afc71ee3f. Latest consensus state: current_term: 1 leader_uuid: "f6993c8e2d624b9891e1879afc71ee3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6993c8e2d624b9891e1879afc71ee3f" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 33011 } } }
I20260812 03:52:03.169240 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:03.169276 10521 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f [sys.catalog]: This master's current role is: LEADER
W20260812 03:52:03.169577 10548 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:03.169641 10548 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:03.170835 10549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:03.171033 10549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:03.172582 10549 catalog_manager.cc:1347] Generated new cluster ID: 35570d9593ec4e1293292f61ceddd800
I20260812 03:52:03.172639 10549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 03:52:03 dist-test-slave-39l8 krb5kdc[10371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:52:03.179409 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:03.168642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50117 (local address 127.7.132.190:33011)
0812 03:52:03.168974 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:03.168984 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:03.169020 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:52:03.169143 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:03.169157 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:03.169211 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:03.170206 (+ 995us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:03.170311 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:03.171628 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:03.172589 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:03.173045 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:03.173170 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:03.175520 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:03.175524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:03.175524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:03.175534 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:03.177771 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:03.178189 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:03.178192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:03.178193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:03.178315 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:03.178543 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:03.178544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:03.178544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:03.178824 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:03.178977 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:03.179088 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:52:03.179215 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":62,"threads_started":1}
I20260812 03:52:03.179713 10549 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:03.180311 10549 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:03.180429 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:03.189270 10549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6993c8e2d624b9891e1879afc71ee3f: Generated new TSK 0
I20260812 03:52:03.189411 10549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:03.194837 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:03.189153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34713 (local address 127.7.132.190:33011)
0812 03:52:03.189186 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:03.189188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:03.189325 (+ 137us) server_negotiation.cc:407] Connection header received
0812 03:52:03.189417 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:03.189417 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:03.189426 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:03.189473 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:03.189919 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:03.190376 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:03.191089 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:03.191272 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:03.191342 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:03.192419 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:03.192420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:03.192420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:03.192428 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:03.193715 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:03.194018 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:03.194019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:03.194019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:03.194086 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:03.194302 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:03.194302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:03.194303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:03.194397 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:03.194530 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:03.194641 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:52:03.194747 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:52:03.196579 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10392
2026-08-12T03:52:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:03 dist-test-slave-39l8 krb5kdc[10566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:03 dist-test-slave-39l8 krb5kdc[10566](info): set up 2 sockets
Aug 12 03:52:03 dist-test-slave-39l8 krb5kdc[10566](info): commencing operation
krb5kdc: starting...
W20260812 03:52:04.373087 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:52:04 dist-test-slave-39l8 krb5kdc[10566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:52:04.485128 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37345
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:37345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:04.560678 10588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:04.560899 10588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:04.560935 10588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:04.562005 10588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:04.562070 10588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:04.562094 10588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:04.562124 10588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:04.562171 10588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:04.563735 10588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:37345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:37345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:04.564005 10588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:04.564222 10588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:04.568862 10598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:04.568821 10594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:04.568863 10595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:04.568950 10588 server_base.cc:1034] running on GCE node
I20260812 03:52:04.569236 10588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:04.569466 10588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:04.570603 10588 hybrid_clock.cc:648] HybridClock initialized: now 1786506724570589 us; error 30 us; skew 500 ppm
Aug 12 03:52:04 dist-test-slave-39l8 krb5kdc[10566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506724, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:04.577441 10588 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:52:04.577873 10588 webserver.cc:460] Webserver started at http://127.7.132.190:44939/ using document root <none> and password file <none>
I20260812 03:52:04.578094 10588 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:04.578159 10588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:04.578315 10588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:04.579383 10588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "43e507163c3f4d92bcfb081650ea7707"
format_stamp: "Formatted at 2026-08-12 03:52:04 on dist-test-slave-39l8"
server_key: "f8ae688c50c6e1f6d2fc0f11bdf144f0"
server_key_iv: "de59b9448f687c8518ad92bfe597b462"
server_key_version: "encryptionkey@0"
I20260812 03:52:04.579741 10588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "43e507163c3f4d92bcfb081650ea7707"
format_stamp: "Formatted at 2026-08-12 03:52:04 on dist-test-slave-39l8"
server_key: "f8ae688c50c6e1f6d2fc0f11bdf144f0"
server_key_iv: "de59b9448f687c8518ad92bfe597b462"
server_key_version: "encryptionkey@0"
I20260812 03:52:04.581029 10588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 03:52:04.581710 10610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:04.581871 10588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:04.581974 10588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "43e507163c3f4d92bcfb081650ea7707"
format_stamp: "Formatted at 2026-08-12 03:52:04 on dist-test-slave-39l8"
server_key: "f8ae688c50c6e1f6d2fc0f11bdf144f0"
server_key_iv: "de59b9448f687c8518ad92bfe597b462"
server_key_version: "encryptionkey@0"
I20260812 03:52:04.582043 10588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:04.594131 10588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:04.595865 10588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:04.596062 10588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:04.600040 10588 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:37345
I20260812 03:52:04.600072 10693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:37345 every 8 connection(s)
I20260812 03:52:04.600531 10588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:04.601426 10694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:04.603677 10694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707: Bootstrap starting.
I20260812 03:52:04.604241 10694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:04.604532 10694 log.cc:824] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:04.605242 10694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707: No bootstrap required, opened a new log
I20260812 03:52:04.606799 10694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e507163c3f4d92bcfb081650ea7707" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37345 } }
I20260812 03:52:04.606945 10694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:04.606966 10694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43e507163c3f4d92bcfb081650ea7707, State: Initialized, Role: FOLLOWER
I20260812 03:52:04.607136 10694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e507163c3f4d92bcfb081650ea7707" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37345 } }
I20260812 03:52:04.607214 10694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:04.607254 10694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:04.607301 10694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:04.608155 10694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e507163c3f4d92bcfb081650ea7707" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37345 } }
I20260812 03:52:04.608242 10694 leader_election.cc:302] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43e507163c3f4d92bcfb081650ea7707; no voters:
I20260812 03:52:04.608441 10694 leader_election.cc:288] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:04.608495 10700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:04.608672 10700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [term 1 LEADER]: Becoming Leader. State: Replica: 43e507163c3f4d92bcfb081650ea7707, State: Running, Role: LEADER
I20260812 03:52:04.608711 10694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:04.608759 10700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e507163c3f4d92bcfb081650ea7707" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37345 } }
I20260812 03:52:04.609112 10701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43e507163c3f4d92bcfb081650ea7707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e507163c3f4d92bcfb081650ea7707" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37345 } } }
I20260812 03:52:04.609122 10702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43e507163c3f4d92bcfb081650ea7707. Latest consensus state: current_term: 1 leader_uuid: "43e507163c3f4d92bcfb081650ea7707" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43e507163c3f4d92bcfb081650ea7707" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 37345 } } }
I20260812 03:52:04.609236 10701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:04.609405 10702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:04.609697 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10588
I20260812 03:52:04.609807 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:52:04.609949 7698 external_mini_cluster.cc:1428] Setting key d28442a67aeccbdcf8d6253b97db6eda
W20260812 03:52:04.610514 10725 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:04.610569 10725 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:04.610610 10712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:04.610795 10712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:04.613325 10712 catalog_manager.cc:1347] Generated new cluster ID: d4541676f25d48b0ab44c83435b60fc6
I20260812 03:52:04.613385 10712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:04.615628 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:04.610397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50505 (local address 127.7.132.190:37345)
0812 03:52:04.610545 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:04.610553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:04.610992 (+ 439us) server_negotiation.cc:407] Connection header received
0812 03:52:04.611326 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:04.611350 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:04.611407 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:04.611525 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:04.611913 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:04.612830 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:04.613832 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:04.614227 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:04.614336 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:04.615025 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:04.615029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:04.615029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:04.615191 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:04.615218 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:04.615228 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:04.615296 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:04.615362 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 03:52:04.615420 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260812 03:52:04.616609 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:04.620289 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:04.617846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45483 (local address 127.7.132.190:37345)
0812 03:52:04.617882 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:04.617884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:04.617939 (+ 55us) server_negotiation.cc:407] Connection header received
0812 03:52:04.617999 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:04.618000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:04.618009 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:04.618058 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:04.618181 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:04.618629 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:04.619261 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:04.619462 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:04.619531 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:04.620090 (+ 559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:04.620091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:04.620092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:04.620096 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:04.620107 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:04.620110 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:04.620152 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:04.620203 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:52:04.620222 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:52:04.646867 10712 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:04.647348 10712 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:04.652212 10712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43e507163c3f4d92bcfb081650ea7707: Generated new TSK 0
I20260812 03:52:04.652392 10712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:04.659183 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10588
2026-08-12T03:52:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:04 dist-test-slave-39l8 krb5kdc[10745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:04 dist-test-slave-39l8 krb5kdc[10745](info): set up 2 sockets
Aug 12 03:52:04 dist-test-slave-39l8 krb5kdc[10745](info): commencing operation
krb5kdc: starting...
W20260812 03:52:05.829173 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:52:05 dist-test-slave-39l8 krb5kdc[10745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:52:05.941177 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41177
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:33383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:41177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:06.020179 10760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:06.020535 10760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:06.020622 10760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:06.022339 10760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:06.022457 10760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:06.022538 10760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:06.022605 10760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:06.022667 10760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:06.024874 10760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:33383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:41177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:06.025246 10760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:06.025494 10760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:06.030475 10775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:06.030498 10760 server_base.cc:1034] running on GCE node
W20260812 03:52:06.030488 10772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:06.030483 10778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:06.030902 10760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:06.031198 10760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:06.032366 10760 hybrid_clock.cc:648] HybridClock initialized: now 1786506726032359 us; error 38 us; skew 500 ppm
Aug 12 03:52:06 dist-test-slave-39l8 krb5kdc[10745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506726, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:06.035223 10760 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:52:06.035673 10760 webserver.cc:460] Webserver started at http://127.7.132.190:45057/ using document root <none> and password file <none>
I20260812 03:52:06.035858 10760 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:06.035898 10760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:06.035981 10760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:06.036963 10760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "191f048723a343c0994a033da88a299b"
format_stamp: "Formatted at 2026-08-12 03:52:06 on dist-test-slave-39l8"
server_key: "62c9374d07cb78dc1337de319fe8aa91"
server_key_iv: "e818d2d281616c6e864ac29383e85a10"
server_key_version: "encryptionkey@0"
I20260812 03:52:06.037248 10760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "191f048723a343c0994a033da88a299b"
format_stamp: "Formatted at 2026-08-12 03:52:06 on dist-test-slave-39l8"
server_key: "62c9374d07cb78dc1337de319fe8aa91"
server_key_iv: "e818d2d281616c6e864ac29383e85a10"
server_key_version: "encryptionkey@0"
I20260812 03:52:06.038422 10760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:52:06.039093 10788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:06.039240 10760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:06.039330 10760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "191f048723a343c0994a033da88a299b"
format_stamp: "Formatted at 2026-08-12 03:52:06 on dist-test-slave-39l8"
server_key: "62c9374d07cb78dc1337de319fe8aa91"
server_key_iv: "e818d2d281616c6e864ac29383e85a10"
server_key_version: "encryptionkey@0"
I20260812 03:52:06.039381 10760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:06.042436 10760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:06.044152 10760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:06.044263 10760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:06.048044 10760 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:41177
I20260812 03:52:06.048126 10870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:41177 every 8 connection(s)
I20260812 03:52:06.048401 10760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:06.049372 10875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:06.051591 10875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b: Bootstrap starting.
I20260812 03:52:06.052168 10875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:06.052459 10875 log.cc:824] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:06.053164 10875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b: No bootstrap required, opened a new log
I20260812 03:52:06.054811 10875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191f048723a343c0994a033da88a299b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41177 } }
I20260812 03:52:06.054962 10875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:52:06.055029 10875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191f048723a343c0994a033da88a299b, State: Initialized, Role: FOLLOWER
I20260812 03:52:06.055189 10875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191f048723a343c0994a033da88a299b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41177 } }
I20260812 03:52:06.055271 10875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:52:06.055315 10875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:52:06.055363 10875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:06.055600 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10760
I20260812 03:52:06.055714 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:52:06.055907 7698 external_mini_cluster.cc:1428] Setting key 48e31d672de152f6391df41bb5c280bb
I20260812 03:52:06.056326 10875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191f048723a343c0994a033da88a299b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41177 } }
I20260812 03:52:06.056466 10875 leader_election.cc:302] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 191f048723a343c0994a033da88a299b; no voters:
I20260812 03:52:06.056618 10875 leader_election.cc:288] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:52:06.056680 10884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:06.056927 10884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [term 1 LEADER]: Becoming Leader. State: Replica: 191f048723a343c0994a033da88a299b, State: Running, Role: LEADER
I20260812 03:52:06.057053 10875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:06.057050 10884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191f048723a343c0994a033da88a299b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41177 } }
I20260812 03:52:06.058305 10886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 191f048723a343c0994a033da88a299b. Latest consensus state: current_term: 1 leader_uuid: "191f048723a343c0994a033da88a299b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191f048723a343c0994a033da88a299b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41177 } } }
I20260812 03:52:06.058373 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:06.056324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48199 (local address 127.7.132.190:41177)
0812 03:52:06.056498 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:06.056506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:06.056537 (+ 31us) server_negotiation.cc:407] Connection header received
0812 03:52:06.056684 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:06.056691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:06.056754 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:06.056905 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:06.057905 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:06.057908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:06.057909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:06.057913 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:06.057965 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:06.057979 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:06.058055 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:06.058133 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 03:52:06.058182 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260812 03:52:06.058395 10886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:06.058768 10885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "191f048723a343c0994a033da88a299b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191f048723a343c0994a033da88a299b" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41177 } } }
I20260812 03:52:06.058871 10885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b [sys.catalog]: This master's current role is: LEADER
W20260812 03:52:06.058957 10903 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:06.059020 10903 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:06.059123 10904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:06.059342 10904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:06.059343 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:06.061677 10882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:06.060751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53135 (local address 127.7.132.190:41177)
0812 03:52:06.060789 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:06.060791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:06.060898 (+ 107us) server_negotiation.cc:407] Connection header received
0812 03:52:06.060936 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:06.060937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:06.060941 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:06.060982 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:06.061456 (+ 474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:06.061457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:06.061457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:52:06.061460 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:06.061471 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:52:06.061474 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:06.061512 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:06.061566 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:52:06.061575 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:52:06.063087 10904 catalog_manager.cc:1347] Generated new cluster ID: 87a9f6b591e54d87847cf80fdae39566
I20260812 03:52:06.063143 10904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:06.072975 10904 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:06.073127 10904 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:06.083462 10904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 191f048723a343c0994a033da88a299b: Generated new TSK 0
I20260812 03:52:06.083688 10904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:06.092423 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10760
2026-08-12T03:52:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1437 ms)
[----------] 18 tests from AuthTokenIssuingTest (25942 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:06 dist-test-slave-39l8 krb5kdc[10925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:06 dist-test-slave-39l8 krb5kdc[10925](info): set up 2 sockets
Aug 12 03:52:06 dist-test-slave-39l8 krb5kdc[10925](info): commencing operation
krb5kdc: starting...
W20260812 03:52:07.281358 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:52:07.391605 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41237
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:41237,127.7.132.189:35683,127.7.132.188:44925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:07.467850 10946 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:52:07.468048 10946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:07.468081 10946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:07.468116 10946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:07.469174 10946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:07.469234 10946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:07.469257 10946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:07.469278 10946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:07.469305 10946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:07.470688 10946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:41237,127.7.132.189:35683,127.7.132.188:44925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:41237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:07.471010 10946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:07.471225 10946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:07.475911 10958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:07.476017 10946 server_base.cc:1034] running on GCE node
W20260812 03:52:07.475937 10955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:07.475934 10954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:07.476361 10946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:07.476634 10946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:07.477784 10946 hybrid_clock.cc:648] HybridClock initialized: now 1786506727477768 us; error 42 us; skew 500 ppm
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:07.480082 10946 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:52:07.480516 10946 webserver.cc:460] Webserver started at http://127.7.132.190:38365/ using document root <none> and password file <none>
I20260812 03:52:07.480759 10946 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:07.480826 10946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:07.480979 10946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:07.481961 10946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "07f9020a1e4f485091fd3f31ba243fb3"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "608865d75045b9291297d27394cf5563"
server_key_iv: "1349b55f047f1f816c67cf15d9de8244"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.482287 10946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "07f9020a1e4f485091fd3f31ba243fb3"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "608865d75045b9291297d27394cf5563"
server_key_iv: "1349b55f047f1f816c67cf15d9de8244"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.483680 10946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 03:52:07.484390 10967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:07.484547 10946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:52:07.484660 10946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "07f9020a1e4f485091fd3f31ba243fb3"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "608865d75045b9291297d27394cf5563"
server_key_iv: "1349b55f047f1f816c67cf15d9de8244"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.484735 10946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:07.494235 10946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:07.495724 10946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:07.495858 10946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:07.499519 10946 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:41237
I20260812 03:52:07.499547 11054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:41237 every 8 connection(s)
I20260812 03:52:07.499850 10946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:07.500723 11056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:07.501920 11056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:07.506361 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 10946
I20260812 03:52:07.506472 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:52:07.506659 7698 external_mini_cluster.cc:1428] Setting key 4aa24ffd7a6f930338bdf859bee57f49
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:52:07.511121 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.502468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.7.132.190:41237)
0812 03:52:07.502622 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.502629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.502769 (+ 140us) server_negotiation.cc:407] Connection header received
0812 03:52:07.503297 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.503299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.503315 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.503410 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.503916 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.504678 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.505534 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.505868 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.506006 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.508053 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.508060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.508060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.508067 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.509551 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.509939 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.509942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.509942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.510021 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.510258 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.510259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.510260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.510445 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.510532 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.510753 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.510957 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260812 03:52:07.511123 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.502409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41237 (local address 127.0.0.1:32804)
0812 03:52:07.502616 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.502670 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.503136 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.503415 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.503418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.503669 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.503801 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.503805 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.504804 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.504805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.505439 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.505441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.505997 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.505998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.506101 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.506629 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.506638 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.507955 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.509652 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.509658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.509660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.509845 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.510089 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.510089 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.510090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.510176 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.510516 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.510521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.510579 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.510683 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.510908 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":27,"thread_start_us":86,"threads_started":1}
I20260812 03:52:07.511934 11056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:07.512234 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.512127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49882)
0812 03:52:07.512171 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.512215 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:52:07.512354 10972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
I20260812 03:52:07.512768 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.512665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49892)
0812 03:52:07.512710 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.512741 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 03:52:07.512862 11056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:35683: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.132.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
I20260812 03:52:07.529254 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:35683
--webserver_interface=127.7.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:41237,127.7.132.189:35683,127.7.132.188:44925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:52:07.562191 11056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } attempt: 1
I20260812 03:52:07.562649 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.562528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49902)
0812 03:52:07.562585 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.562622 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 03:52:07.563138 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.562993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49912)
0812 03:52:07.563054 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.563099 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 03:52:07.563295 11056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:35683: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
W20260812 03:52:07.606846 11066 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:52:07.607023 11066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:07.607046 11066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:07.607091 11066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:07.608126 11066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:07.608183 11066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:07.608197 11066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:07.608209 11066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:07.608253 11066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:07.609671 11066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:41237,127.7.132.189:35683,127.7.132.188:44925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:35683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:07.609899 11066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:07.610147 11066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:07.614681 11076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:07.614675 11079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:07.614786 11074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:07.615013 11066 server_base.cc:1034] running on GCE node
I20260812 03:52:07.615216 11066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:07.615474 11066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:07.616622 11066 hybrid_clock.cc:648] HybridClock initialized: now 1786506727616616 us; error 35 us; skew 500 ppm
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:07.619019 11066 init.cc:377] Logged in from keytab as kudu/127.7.132.189@KRBTEST.COM (short username kudu)
I20260812 03:52:07.619370 11066 webserver.cc:460] Webserver started at http://127.7.132.189:46453/ using document root <none> and password file <none>
I20260812 03:52:07.619556 11066 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:07.619594 11066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:07.619686 11066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:07.620625 11066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/data/instance:
uuid: "1beabc54f8214ba580c0598d17af9295"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "17888dc52a317c920902e44becd4c645"
server_key_iv: "7a99e291ad8a609362111f886451fc5d"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.620922 11066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/wal/instance:
uuid: "1beabc54f8214ba580c0598d17af9295"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "17888dc52a317c920902e44becd4c645"
server_key_iv: "7a99e291ad8a609362111f886451fc5d"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.622020 11066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 03:52:07.622673 11087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:07.622869 11066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 03:52:07.622956 11066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/wal
uuid: "1beabc54f8214ba580c0598d17af9295"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "17888dc52a317c920902e44becd4c645"
server_key_iv: "7a99e291ad8a609362111f886451fc5d"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.623009 11066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:07.631459 11056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } attempt: 2
I20260812 03:52:07.631872 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.631736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49928)
0812 03:52:07.631799 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.631838 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 03:52:07.632254 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.632167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49938)
0812 03:52:07.632212 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.632236 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:52:07.632349 11056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:35683: Network error: Client connection negotiation failed: client connection to 127.7.132.189:35683: connect: Connection refused (error 111)
I20260812 03:52:07.640087 11066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:07.641785 11066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:07.641943 11066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:07.645750 11066 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:35683
I20260812 03:52:07.645807 11171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:35683 every 8 connection(s)
I20260812 03:52:07.646139 11066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
I20260812 03:52:07.647123 11172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:07.648530 11172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:07.653491 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 11066
I20260812 03:52:07.653614 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-1/wal/instance
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:52:07.653831 7698 external_mini_cluster.cc:1428] Setting key 3da2a7ef001b56b82328ce61c6feec6f
I20260812 03:52:07.657274 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.649103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41237 (local address 127.0.0.1:32808)
0812 03:52:07.649281 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.649318 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.649722 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.649910 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.649913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.650127 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.650311 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.650315 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.650982 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.650983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.651726 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.651728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.652064 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.652065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.652164 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.652752 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.652760 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.654284 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.655867 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.655874 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.655877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.656180 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.656368 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.656369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.656369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.656493 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.656720 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.656744 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.656885 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.656998 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.657115 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260812 03:52:07.657272 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.649168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.7.132.190:41237)
0812 03:52:07.649224 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.649226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.649394 (+ 168us) server_negotiation.cc:407] Connection header received
0812 03:52:07.649796 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.649797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.649809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.649856 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.650378 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.650905 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.651797 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.651975 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.652034 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.654387 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.654389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.654389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.654396 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.655786 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.656260 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.656261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.656261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.656309 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.656559 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.656560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.656560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.656654 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.656707 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.657047 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.657135 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 03:52:07.658020 11172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.132.188@KRBTEST.COM; defaulting to no policy
I20260812 03:52:07.664958 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.658323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49952)
0812 03:52:07.658388 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.658396 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.658429 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.658775 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.658776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.658910 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.658956 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.658957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.659757 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.659758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.660231 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.660233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.660683 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.660684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.660752 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.661165 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.661168 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.662411 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.664095 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.664095 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.664096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.664245 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.664417 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.664417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.664418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.664463 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.664740 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.664741 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.664775 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.664829 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.664906 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 03:52:07.665093 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.658407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.7.132.189:35683)
0812 03:52:07.658591 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.658595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.658637 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:52:07.658661 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.658663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.658676 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.658773 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.659036 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.659670 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.660322 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.660607 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.660687 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.662487 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.662493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.662494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.662499 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.664024 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.664314 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.664316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.664316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.664361 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.664552 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.664552 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.664552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.664683 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.664742 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.664861 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.664947 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.188@KRBTEST.COM" created.
I20260812 03:52:07.665639 11172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:07.665943 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.665833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44326)
0812 03:52:07.665878 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.665916 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:52:07.666097 11090 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
I20260812 03:52:07.666502 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.666425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44330)
0812 03:52:07.666465 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.666487 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:52:07.666616 11172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:44925: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
I20260812 03:52:07.679070 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:44925
--webserver_interface=127.7.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:39779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:41237,127.7.132.189:35683,127.7.132.188:44925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:52:07.715826 11172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } attempt: 1
I20260812 03:52:07.716331 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.716202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44334)
0812 03:52:07.716267 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.716304 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 03:52:07.716672 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.716568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44350)
0812 03:52:07.716615 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.716640 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:52:07.716778 11172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:44925: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
I20260812 03:52:07.723475 11056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } attempt: 3
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:52:07.730469 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.723852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49962 (local address 127.7.132.189:35683)
0812 03:52:07.723926 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.723928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.723968 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:52:07.723997 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.723998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.724010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.724066 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.724362 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.725082 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.725791 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.726008 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.726108 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.728033 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.728035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.728036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.728041 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.729388 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.729734 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.729735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.729735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.729790 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.729981 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.729982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.729983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.730086 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.730161 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.730326 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.730394 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:52:07.730470 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.723763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49962)
0812 03:52:07.723822 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.723835 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.723899 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.724079 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.724080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.724225 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.724289 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.724291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.725173 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.725175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.725705 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.725707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.726116 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.726117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.726200 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.726615 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.726621 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.727935 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.729487 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.729488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.729488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.729651 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.729859 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.729860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.729860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.729923 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.730174 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.730175 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.730216 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.730291 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.730384 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:52:07.730803 11056 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:07.731106 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.731015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44364)
0812 03:52:07.731061 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.731085 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 03:52:07.731457 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.731381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44368)
0812 03:52:07.731421 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.731444 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:52:07.731551 11056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:44925: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
W20260812 03:52:07.757257 11186 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:52:07.757438 11186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:07.757455 11186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:07.757501 11186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:07.758612 11186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:07.758675 11186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:07.758689 11186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:07.758760 11186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:07.758811 11186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:07.760242 11186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:39779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:41237,127.7.132.189:35683,127.7.132.188:44925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:44925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:07.760483 11186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:07.760704 11186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:52:07.762742 11056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } attempt: 1
I20260812 03:52:07.763165 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.763033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44380)
0812 03:52:07.763103 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.763139 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:52:07.763509 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.763419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44384)
0812 03:52:07.763466 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.763491 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 03:52:07.763613 11056 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:44925: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
W20260812 03:52:07.765564 11204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:07.765589 11199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:07.765595 11197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:07.765806 11186 server_base.cc:1034] running on GCE node
I20260812 03:52:07.766036 11186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:07.766333 11186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:07.767473 11186 hybrid_clock.cc:648] HybridClock initialized: now 1786506727767440 us; error 46 us; skew 500 ppm
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:07.769820 11186 init.cc:377] Logged in from keytab as kudu/127.7.132.188@KRBTEST.COM (short username kudu)
I20260812 03:52:07.770277 11186 webserver.cc:460] Webserver started at http://127.7.132.188:41681/ using document root <none> and password file <none>
I20260812 03:52:07.770498 11186 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:07.770565 11186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:07.770696 11186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:07.771713 11186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/data/instance:
uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "16cc71b9667733da030fee7088ec7f00"
server_key_iv: "d325a0a1418b142d5e3fdf19da908c4a"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.772085 11186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/wal/instance:
uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "16cc71b9667733da030fee7088ec7f00"
server_key_iv: "d325a0a1418b142d5e3fdf19da908c4a"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.773388 11186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 03:52:07.774080 11213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:07.774238 11186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 03:52:07.774338 11186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/wal
uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161"
format_stamp: "Formatted at 2026-08-12 03:52:07 on dist-test-slave-39l8"
server_key: "16cc71b9667733da030fee7088ec7f00"
server_key_iv: "d325a0a1418b142d5e3fdf19da908c4a"
server_key_version: "encryptionkey@0"
I20260812 03:52:07.774423 11186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:07.784901 11172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } attempt: 2
I20260812 03:52:07.785266 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.785143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44390)
0812 03:52:07.785205 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.785240 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:52:07.785552 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.785469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44396)
0812 03:52:07.785511 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.785533 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:52:07.785640 11172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:44925: Network error: Client connection negotiation failed: client connection to 127.7.132.188:44925: connect: Connection refused (error 111)
I20260812 03:52:07.796020 11186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:07.797608 11186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:07.797765 11186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:07.801527 11186 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:44925
I20260812 03:52:07.801550 11295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:44925 every 8 connection(s)
I20260812 03:52:07.801869 11186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
I20260812 03:52:07.802865 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:07.803634 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 11186
I20260812 03:52:07.803715 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786506457253237-7698-0/minicluster-data/master-2/wal/instance
I20260812 03:52:07.803902 7698 external_mini_cluster.cc:1428] Setting key 3ce65b934c5d19f02925c45aa2c6552a
I20260812 03:52:07.804719 11296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:52:07.813889 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.805773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.7.132.190:41237)
0812 03:52:07.805834 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.805836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.805980 (+ 144us) server_negotiation.cc:407] Connection header received
0812 03:52:07.806053 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.806054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.806065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.806121 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.806581 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.807237 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.808341 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.808592 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.808682 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.810979 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.810980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.810981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.810987 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.812386 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.812901 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.812902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.812903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.812970 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.813226 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.813227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.813227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.813345 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.813415 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.813703 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.813802 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 03:52:07.814054 11302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.805719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:41237 (local address 127.0.0.1:32818)
0812 03:52:07.805888 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.805919 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.805998 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.806167 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.806171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.806410 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.806493 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.806497 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.807339 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.807340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.808273 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.808276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.808705 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.808706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.808764 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.809182 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.809190 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.810868 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.812453 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.812461 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.812463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.812798 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.813038 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.813039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.813039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.813145 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.813420 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.813428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.813556 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.813676 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.813829 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260812 03:52:07.814574 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.804406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44410 (local address 127.7.132.188:44925)
0812 03:52:07.804587 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.804596 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.804626 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:52:07.805126 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.805135 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.805206 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.805356 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.806119 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.807275 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.808222 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.808601 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.808697 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.811587 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.811596 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.811596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.811603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.813288 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.813641 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.813644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.813645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.813703 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.813922 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.813922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.813923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.814088 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.814209 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.814407 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.814480 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260812 03:52:07.814580 11296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:52:07.820988 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.815050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.7.132.189:35683)
0812 03:52:07.815108 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.815110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.815118 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:07.815141 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.815142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.815153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.815201 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.815521 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.815975 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.816639 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.816815 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.816861 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.818356 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.818357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.818357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.818364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.819801 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.820159 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.820160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.820161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.820220 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.820456 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.820457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.820457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.820594 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.820665 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.820870 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.820931 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:52:07.821014 11302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.814952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:35683 (local address 127.0.0.1:49978)
0812 03:52:07.815001 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.815012 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.815056 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.815224 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.815225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.815369 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.815442 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.815444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.816040 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.816041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.816569 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.816571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.816877 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.816878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.816943 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.817303 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.817307 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.818273 (+ 966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.819902 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.819903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.819903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.820074 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.820297 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.820298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.820298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.820388 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.820677 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.820678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.820736 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.820825 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.820911 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 03:52:07.821676 11296 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:07.823158 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.815585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.7.132.190:41237)
0812 03:52:07.815618 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.815620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.816104 (+ 484us) server_negotiation.cc:407] Connection header received
0812 03:52:07.816124 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.816125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.816133 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.816204 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.816572 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.817099 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.817762 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.817939 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.818003 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.820301 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.820302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.820302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.820309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.821992 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.822406 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.822407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.822407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.822466 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.822666 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.822667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.822667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.822845 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.822952 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.823050 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.823102 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:52:07.828353 11302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.821912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44424)
0812 03:52:07.821998 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.822008 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.822050 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.822213 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.822214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.822327 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.822368 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.822370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.823218 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.823219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.823959 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.823962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.824331 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.824332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.824378 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.824759 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.824764 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.825933 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.827477 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.827478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.827478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.827624 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.827817 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.827817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.827818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.827866 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.828079 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.828080 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.828111 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.828170 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.828254 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 03:52:07.828569 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.821959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44424 (local address 127.7.132.188:44925)
0812 03:52:07.821998 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.822000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.822048 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:52:07.822117 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.822118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.822143 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.822211 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.822458 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.823129 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.824052 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.824254 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.824320 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.826018 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.826019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.826019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.826025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.827384 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.827699 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.827700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.827700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.827751 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.827921 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.827922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.827922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.828014 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.828075 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.828414 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.828475 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:52:07.830168 11296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161: Bootstrap starting.
I20260812 03:52:07.831081 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.823909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.7.132.189:35683)
0812 03:52:07.823947 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.823949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.823957 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:07.823997 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.823998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.824007 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.824059 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.824503 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.824976 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.825850 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.826134 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.826196 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.828186 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.828187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.828187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.828193 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.829812 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.830136 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.830137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.830138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.830205 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.830422 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.830423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.830423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.830556 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.830771 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.830822 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.831010 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:52:07.831245 11296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:07.831604 11296 log.cc:824] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:07.832479 11296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161: No bootstrap required, opened a new log
I20260812 03:52:07.832553 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:07.834347 11296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } }
I20260812 03:52:07.834491 11296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d6aae09b23e4d5c912c8f9dbbbc1161, State: Initialized, Role: FOLLOWER
I20260812 03:52:07.833671 11016 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "07f9020a1e4f485091fd3f31ba243fb3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32820
I20260812 03:52:07.834563 11296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } }
I20260812 03:52:07.834883 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.834925 11296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:07.834972 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:52:07.836884 11327 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:07.836933 11327 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:07.838801 11056 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } attempt: 2
I20260812 03:52:07.836958 11186 master_runner.cc:186] Error getting master registration for 127.7.132.190:41237: OK, instance_id { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" instance_seqno: 1786506727498418 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:52:07.839468 11186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:52:07.844877 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.839039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.7.132.188:44925)
0812 03:52:07.839144 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.839146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.839150 (+ 4us) server_negotiation.cc:407] Connection header received
0812 03:52:07.839164 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.839164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.839170 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.839226 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.839468 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.839900 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.840498 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.840686 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.840761 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.842329 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.842330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.842330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.842336 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.843751 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.844113 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.844114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.844115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.844165 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.844398 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.844398 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.844399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.844512 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.844584 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.844736 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.844799 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 03:52:07.844885 11059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.839008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44430)
0812 03:52:07.839048 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.839056 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.839087 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.839238 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.839239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.839362 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.839388 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.839389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.839980 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.839980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.840407 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.840408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.840761 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.840762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.840812 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.841186 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.841190 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.842245 (+ 1055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.843855 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.843856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.843856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.844035 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.844267 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.844268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.844268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.844322 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.844579 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.844580 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.844621 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.844707 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.844794 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 03:52:07.845999 11056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3: Bootstrap starting.
I20260812 03:52:07.846565 11056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:07.846886 11056 log.cc:824] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:07.847595 11056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3: No bootstrap required, opened a new log
I20260812 03:52:07.848546 11056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } }
I20260812 03:52:07.848637 11056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07f9020a1e4f485091fd3f31ba243fb3, State: Initialized, Role: FOLLOWER
I20260812 03:52:07.848735 11056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } }
I20260812 03:52:07.849050 11056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:07.849012 11332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.849129 11016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:52:07.849263 11332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:07.849331 11016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:07.850116 11016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } }
I20260812 03:52:07.850360 11016 leader_election.cc:288] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [CANDIDATE]: Term 1 election: Requested vote from peers 1beabc54f8214ba580c0598d17af9295 (127.7.132.189:35683), 3d6aae09b23e4d5c912c8f9dbbbc1161 (127.7.132.188:44925)
I20260812 03:52:07.850800 11252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07f9020a1e4f485091fd3f31ba243fb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161"
I20260812 03:52:07.850910 11252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:07.851679 11252 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f9020a1e4f485091fd3f31ba243fb3 in term 1.
I20260812 03:52:07.851872 10972 leader_election.cc:302] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07f9020a1e4f485091fd3f31ba243fb3, 3d6aae09b23e4d5c912c8f9dbbbc1161; no voters:
I20260812 03:52:07.852020 11332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:07.852211 11332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [term 1 LEADER]: Becoming Leader. State: Replica: 07f9020a1e4f485091fd3f31ba243fb3, State: Running, Role: LEADER
I20260812 03:52:07.852291 11332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } }
W20260812 03:52:07.852317 11350 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:07.852532 11350 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:07.852664 11332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07f9020a1e4f485091fd3f31ba243fb3. Latest consensus state: current_term: 1 leader_uuid: "07f9020a1e4f485091fd3f31ba243fb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.852761 11332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:07.851557 10946 master_runner.cc:186] Error getting master registration for 127.7.132.189:35683: OK, instance_id { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" instance_seqno: 1786506727644734 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:52:07.853943 10946 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:52:07.854077 11355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:07.854238 11355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:07.855679 11252 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [term 1 FOLLOWER]: Refusing update from remote peer 07f9020a1e4f485091fd3f31ba243fb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:07.855954 11332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:07.857087 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07f9020a1e4f485091fd3f31ba243fb3. Latest consensus state: current_term: 1 leader_uuid: "07f9020a1e4f485091fd3f31ba243fb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.857193 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:07.867534 11352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "07f9020a1e4f485091fd3f31ba243fb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.867611 11352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:07.867669 11361 mvcc.cc:205] Tried to move back new op lower bound from 7317531557294653440 to 7317531557283586048. Current Snapshot: MvccSnapshot[applied={T|T < 7317531557294653440}]
I20260812 03:52:07.867723 11304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "07f9020a1e4f485091fd3f31ba243fb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.867802 11304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:07.867975 11364 mvcc.cc:205] Tried to move back new op lower bound from 7317531557294653440 to 7317531557283586048. Current Snapshot: MvccSnapshot[applied={T|T < 7317531557294653440}]
I20260812 03:52:07.868964 11180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.859030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.7.132.189:35683)
0812 03:52:07.859077 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.859078 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.859656 (+ 578us) server_negotiation.cc:407] Connection header received
0812 03:52:07.859793 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.859794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.859817 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.859846 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.861062 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.861726 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.863408 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.863728 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.863770 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.865878 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.865880 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.865880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.865886 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.867349 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.867941 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.867942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.867943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.868013 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.868315 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.868328 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.868328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.868432 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.868510 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.868806 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.868878 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 03:52:07.872107 11355 catalog_manager.cc:1347] Generated new cluster ID: 20e17de3790f410685f4a3d385151b5e
I20260812 03:52:07.872155 11355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:07.876754 11172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } attempt: 3
I20260812 03:52:07.884220 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.859205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.7.132.188:44925)
0812 03:52:07.859246 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.859248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.862905 (+ 3657us) server_negotiation.cc:407] Connection header received
0812 03:52:07.863001 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.863002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.863011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.863039 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.864906 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.865501 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.868239 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.868502 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.868537 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.870545 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.870546 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.870546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.870552 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.874017 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.883063 (+ 9046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.883065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.883065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.883132 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.883499 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.883500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.883500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.883628 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.883710 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.884055 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.884128 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:52:07.884258 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.859375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.7.132.190:41237)
0812 03:52:07.859413 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.859415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.863143 (+ 3728us) server_negotiation.cc:407] Connection header received
0812 03:52:07.863932 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.863933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.863941 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.863977 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.866336 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.867184 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.868929 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.869197 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.869255 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.871498 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.871499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.871500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.871506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.873255 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.883352 (+ 10097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.883354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.883354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.883410 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.883685 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.883686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.883687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.883822 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.883921 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.884094 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.884166 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 03:52:07.886930 11355 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:07.887619 11355 catalog_manager.cc:1504] Loading token signing keys...
Aug 12 03:52:07 dist-test-slave-39l8 krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506727, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:52:07.892139 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.876973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:44925 (local address 127.0.0.1:44436)
0812 03:52:07.877039 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:07.877046 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:07.877107 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:07.884434 (+ 7327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:07.884436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:07.884578 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:07.884618 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.884620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.885570 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.885571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.886375 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:07.886377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.886947 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.886948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:07.887014 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.887514 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:07.887519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:07.888869 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:07.890787 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.890788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.890788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.890988 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.891345 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:07.891346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:07.891346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:07.891420 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:07.891830 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:07.891831 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:07.891883 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 03:52:07.891939 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:52:07.892041 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 03:52:07.892702 11299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:07.877211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44436 (local address 127.7.132.188:44925)
0812 03:52:07.884285 (+ 7074us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.884286 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.884293 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:52:07.884309 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.884310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.884318 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.884365 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.884732 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.885479 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.886520 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:07.886823 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:07.886881 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:07.888976 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:07.888977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:07.888977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:07.888983 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:07.890625 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.891086 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.891087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.891087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.891213 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:07.891495 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:07.891496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:07.891496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:07.891700 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:07.892309 (+ 609us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:07.892383 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:52:07.892513 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7062,"thread_start_us":73,"threads_started":1}
I20260812 03:52:07.893776 11355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3: Generated new TSK 0
I20260812 03:52:07.893975 11355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:07.894207 11172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295: Bootstrap starting.
I20260812 03:52:07.894920 11172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:07.895200 11172 log.cc:824] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:07.896106 11172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295: No bootstrap required, opened a new log
I20260812 03:52:07.897915 11172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } }
I20260812 03:52:07.898048 11172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1beabc54f8214ba580c0598d17af9295, State: Initialized, Role: FOLLOWER
I20260812 03:52:07.898121 11172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } }
I20260812 03:52:07.898514 11131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:07.898449 11172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:07.899369 11131 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [term 1 FOLLOWER]: Refusing update from remote peer 07f9020a1e4f485091fd3f31ba243fb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:07.899667 11332 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:07.898486 11133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "07f9020a1e4f485091fd3f31ba243fb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1beabc54f8214ba580c0598d17af9295"
I20260812 03:52:07.898432 11381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.900327 11381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:07.900910 11133 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07f9020a1e4f485091fd3f31ba243fb3 in term 1.
I20260812 03:52:07.903070 11406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "07f9020a1e4f485091fd3f31ba243fb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.903172 11406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:52:07.903652 11407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:07.903704 11407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:07.908667 11381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07f9020a1e4f485091fd3f31ba243fb3. Latest consensus state: current_term: 1 leader_uuid: "07f9020a1e4f485091fd3f31ba243fb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:07.908802 11381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:07.909355 11402 mvcc.cc:205] Tried to move back new op lower bound from 7317531557445644288 to 7317531557283586048. Current Snapshot: MvccSnapshot[applied={T|T < 7317531557294653440}]
W20260812 03:52:07.927672 11061 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.925992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32840 (local address 127.7.132.190:41237)
0812 03:52:07.926056 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.926058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.926969 (+ 911us) server_negotiation.cc:407] Connection header received
0812 03:52:07.927174 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.927175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.927186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.927215 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.927612 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32840: BlockingRecv error: recv got EOF from 127.0.0.1:32840 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 03:52:07.928465 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.926373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50002 (local address 127.7.132.189:35683)
0812 03:52:07.926428 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.926429 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.927125 (+ 696us) server_negotiation.cc:407] Connection header received
0812 03:52:07.928061 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.928062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.928071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.928098 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.928416 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50002: BlockingRecv error: recv got EOF from 127.0.0.1:50002 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 03:52:07.929343 11299 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.926659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.7.132.188:44925)
0812 03:52:07.926696 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.926698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.928784 (+ 2086us) server_negotiation.cc:407] Connection header received
0812 03:52:07.928933 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.928934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.928943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.928969 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.929290 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44450: BlockingRecv error: recv got EOF from 127.0.0.1:44450 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 03:52:07.931521 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.930834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50016 (local address 127.7.132.189:35683)
0812 03:52:07.930866 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.930867 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.931072 (+ 205us) server_negotiation.cc:407] Connection header received
0812 03:52:07.931172 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.931172 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.931187 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.931213 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.931482 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50016: BlockingRecv error: recv got EOF from 127.0.0.1:50016 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 03:52:07.934516 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.933687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.7.132.189:35683)
0812 03:52:07.933718 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.933720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.934074 (+ 354us) server_negotiation.cc:407] Connection header received
0812 03:52:07.934160 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.934161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.934169 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.934193 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.934464 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50026: BlockingRecv error: recv got EOF from 127.0.0.1:50026 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 03:52:07.938294 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.937629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.7.132.189:35683)
0812 03:52:07.937657 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.937659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.937865 (+ 206us) server_negotiation.cc:407] Connection header received
0812 03:52:07.937946 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.937947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.937955 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.937979 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.938262 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50028: BlockingRecv error: recv got EOF from 127.0.0.1:50028 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260812 03:52:07.944128 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.943364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50038 (local address 127.7.132.189:35683)
0812 03:52:07.943397 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.943398 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.943636 (+ 238us) server_negotiation.cc:407] Connection header received
0812 03:52:07.943733 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.943734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.943743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.943771 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.944090 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50038: BlockingRecv error: recv got EOF from 127.0.0.1:50038 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 03:52:07.953966 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.953199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.7.132.189:35683)
0812 03:52:07.953242 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.953243 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.953478 (+ 235us) server_negotiation.cc:407] Connection header received
0812 03:52:07.953575 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.953576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.953585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.953612 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.953921 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50050: BlockingRecv error: recv got EOF from 127.0.0.1:50050 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 03:52:07.971956 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:07.971187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.7.132.189:35683)
0812 03:52:07.971221 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:07.971223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:07.971461 (+ 238us) server_negotiation.cc:407] Connection header received
0812 03:52:07.971551 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:07.971552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:07.971561 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:07.971590 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:07.971909 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50056: BlockingRecv error: recv got EOF from 127.0.0.1:50056 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 03:52:08.007314 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:08.005312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50068 (local address 127.7.132.189:35683)
0812 03:52:08.005356 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.005358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.006627 (+ 1269us) server_negotiation.cc:407] Connection header received
0812 03:52:08.006782 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.006784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.006798 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.006829 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:08.007255 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50068: BlockingRecv error: recv got EOF from 127.0.0.1:50068 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 03:52:08.073252 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:08.072567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.7.132.189:35683)
0812 03:52:08.072639 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.072642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.072748 (+ 106us) server_negotiation.cc:407] Connection header received
0812 03:52:08.072808 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.072809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.072820 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.072887 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:08.073204 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50074: BlockingRecv error: recv got EOF from 127.0.0.1:50074 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:52:08.203254 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:08.202456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50076 (local address 127.7.132.189:35683)
0812 03:52:08.202546 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.202548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.202594 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:52:08.202661 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.202662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.202676 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.202799 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:08.203172 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50076: BlockingRecv error: recv got EOF from 127.0.0.1:50076 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:52:08.318727 11332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "07f9020a1e4f485091fd3f31ba243fb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07f9020a1e4f485091fd3f31ba243fb3" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 41237 } } peers { permanent_uuid: "1beabc54f8214ba580c0598d17af9295" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 35683 } } peers { permanent_uuid: "3d6aae09b23e4d5c912c8f9dbbbc1161" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 44925 } } }
I20260812 03:52:08.318898 11332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07f9020a1e4f485091fd3f31ba243fb3 [sys.catalog]: This master's current role is: LEADER
W20260812 03:52:08.461150 11180 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:08.460490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50090 (local address 127.7.132.189:35683)
0812 03:52:08.460562 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.460564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.460578 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:52:08.460660 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.460661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.460673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.460726 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:08.461059 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50090: BlockingRecv error: recv got EOF from 127.0.0.1:50090 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 03:52:08.837352 11327 catalog_manager.cc:1259] Loaded cluster ID: 20e17de3790f410685f4a3d385151b5e
I20260812 03:52:08.837419 11327 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161: loading cluster ID for follower catalog manager: success
I20260812 03:52:08.838147 11327 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161: acquiring CA information for follower catalog manager: success
I20260812 03:52:08.838431 11327 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d6aae09b23e4d5c912c8f9dbbbc1161: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:52:08.904134 11407 catalog_manager.cc:1259] Loaded cluster ID: 20e17de3790f410685f4a3d385151b5e
I20260812 03:52:08.904232 11407 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295: loading cluster ID for follower catalog manager: success
I20260812 03:52:08.905020 11407 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295: acquiring CA information for follower catalog manager: success
I20260812 03:52:08.905326 11407 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1beabc54f8214ba580c0598d17af9295: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:52:08.976562 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:08.974271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.7.132.189:35683)
0812 03:52:08.974408 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.974410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.974427 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:52:08.974495 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.974495 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.974547 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.974620 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:08.974807 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.975266 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.975933 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.976107 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.976173 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:08.976280 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:08.976382 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:08.976437 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:08.976482 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 03:52:08.976499 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260812 03:52:08.980293 11551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:08.978215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.7.132.188:44925)
0812 03:52:08.978322 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.978324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.978331 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:52:08.978410 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.978411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.978445 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.978511 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:08.978606 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.979118 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.979758 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.979917 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.979961 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:08.980073 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:08.980163 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:08.980213 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:08.980224 (+ 11us) server_negotiation.cc:299] Negotiation successful
0812 03:52:08.980239 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":60,"threads_started":1}
I20260812 03:52:08.984577 11551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:08.981965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.7.132.188:44925)
0812 03:52:08.982012 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.982014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.982091 (+ 77us) server_negotiation.cc:407] Connection header received
0812 03:52:08.982183 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.982184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.982193 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.982243 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:08.982369 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.983083 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.983847 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.984122 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.984178 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:08.984332 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:08.984361 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:08.984411 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:08.984472 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 03:52:08.984493 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:52:08.984654 11540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:08.981736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.7.132.189:35683)
0812 03:52:08.981778 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.981780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.981945 (+ 165us) server_negotiation.cc:407] Connection header received
0812 03:52:08.982079 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.982080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.982089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.982115 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:08.982342 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.982868 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.983913 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.984197 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.984264 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:08.984417 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:08.984443 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:08.984470 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:08.984568 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 03:52:08.984587 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:52:08.984993 11561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:08.981705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.7.132.190:41237)
0812 03:52:08.981842 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:08.981844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:08.981853 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:08.981950 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:08.981951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:08.982002 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:08.982077 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:08.982209 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.983034 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.984081 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:08.984376 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:08.984449 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:08.984655 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:08.984762 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:08.984809 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:08.984889 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:52:08.984914 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":47,"threads_started":1}
I20260812 03:52:08.986696 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 10946
I20260812 03:52:08.998678 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 11066
I20260812 03:52:09.008452 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 11186
2026-08-12T03:52:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2908 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:52:09 dist-test-slave-39l8 krb5kdc[11568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:52:09 dist-test-slave-39l8 krb5kdc[11568](info): set up 2 sockets
Aug 12 03:52:09 dist-test-slave-39l8 krb5kdc[11568](info): commencing operation
krb5kdc: starting...
W20260812 03:52:10.189934 7698 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/test-user.keytab.
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:52:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:52:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.132.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
Entry for principal kudu/127.7.132.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab.
I20260812 03:52:10.298666 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42811
--webserver_interface=127.7.132.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42811,127.7.132.189:36419,127.7.132.188:43053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:52:10.375484 11593 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:52:10.375690 11593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:10.375730 11593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:10.375754 11593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:10.376775 11593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:10.376808 11593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:10.376822 11593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:10.376834 11593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:10.376845 11593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:10.378152 11593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42811,127.7.132.189:36419,127.7.132.188:43053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.190.keytab
--principal=kudu/127.7.132.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.190:42811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.132.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:10.378350 11593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:10.378522 11593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:10.383029 11602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:10.383102 11606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:10.383097 11593 server_base.cc:1034] running on GCE node
W20260812 03:52:10.383038 11604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:10.383461 11593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:10.383742 11593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:10.384895 11593 hybrid_clock.cc:648] HybridClock initialized: now 1786506730384883 us; error 45 us; skew 500 ppm
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:10.390846 11593 init.cc:377] Logged in from keytab as kudu/127.7.132.190@KRBTEST.COM (short username kudu)
I20260812 03:52:10.391194 11593 webserver.cc:460] Webserver started at http://127.7.132.190:39829/ using document root <none> and password file <none>
I20260812 03:52:10.391363 11593 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:10.391400 11593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:10.391479 11593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:10.392407 11593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/data/instance:
uuid: "92c99617e9154477af1ee6403c45d5f2"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "9c367d6804c0b64537aa9e060c9a383c"
server_key_iv: "6a8afc880fd26d830de2423136fe529a"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.392719 11593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/wal/instance:
uuid: "92c99617e9154477af1ee6403c45d5f2"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "9c367d6804c0b64537aa9e060c9a383c"
server_key_iv: "6a8afc880fd26d830de2423136fe529a"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.393930 11593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:10.394563 11618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:10.394735 11593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:10.394820 11593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/wal
uuid: "92c99617e9154477af1ee6403c45d5f2"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "9c367d6804c0b64537aa9e060c9a383c"
server_key_iv: "6a8afc880fd26d830de2423136fe529a"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.394871 11593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:10.403121 11593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:10.404767 11593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:10.404892 11593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:10.408581 11593 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.190:42811
I20260812 03:52:10.408632 11706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.190:42811 every 8 connection(s)
I20260812 03:52:10.408949 11593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/data/info.pb
I20260812 03:52:10.409899 11707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:10.411414 11707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:10.412698 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 11593
I20260812 03:52:10.412782 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-0/wal/instance
I20260812 03:52:10.413007 7698 external_mini_cluster.cc:1428] Setting key b61c57422eea9c6f1d80b42c26b01216
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.132.189@KRBTEST.COM; defaulting to no policy
I20260812 03:52:10.421169 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.412018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42811 (local address 127.0.0.1:59842)
0812 03:52:10.412198 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.412228 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.412647 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.413048 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.413051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.413417 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.413630 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.413636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.414604 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.414606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.415299 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.415301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.415987 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.415988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.416051 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.416525 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.416533 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.417944 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.419722 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.419728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.419730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.419917 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.420162 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.420162 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.420163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.420263 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.420602 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.420607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.420668 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.420765 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.420998 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":101,"thread_start_us":96,"threads_started":1}
I20260812 03:52:10.421172 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.412055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.7.132.190:42811)
0812 03:52:10.412800 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.412804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.412822 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:52:10.412900 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.412901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.412935 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.413002 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.413744 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.414509 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.415373 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.415864 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.415998 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.418032 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.418040 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.418040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.418047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.419581 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.420008 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.420011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.420012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.420087 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.420329 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.420329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.420330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.420525 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.420608 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.420800 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.420936 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":76,"threads_started":1}
I20260812 03:52:10.421900 11707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:10.422212 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.422120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42030)
0812 03:52:10.422156 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.422195 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 03:52:10.422333 11627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
I20260812 03:52:10.422744 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.422637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42044)
0812 03:52:10.422675 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.422698 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 03:52:10.422940 11707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:36419: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
Entry for principal kudu/127.7.132.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab.
I20260812 03:52:10.437253 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:36419
--webserver_interface=127.7.132.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42811,127.7.132.189:36419,127.7.132.188:43053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:52:10.472220 11707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } attempt: 1
I20260812 03:52:10.472806 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.472657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42046)
0812 03:52:10.472733 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.472777 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 03:52:10.473197 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.473110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42062)
0812 03:52:10.473157 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.473182 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 03:52:10.473300 11707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:36419: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
W20260812 03:52:10.521380 11720 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:52:10.521704 11720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:10.521773 11720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:10.521827 11720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:10.523425 11720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:10.523522 11720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:10.523569 11720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:10.523612 11720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:10.523653 11720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:10.525597 11720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42811,127.7.132.189:36419,127.7.132.188:43053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.189.keytab
--principal=kudu/127.7.132.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.189:36419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.132.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:10.525861 11720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:10.526074 11720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:10.532236 11735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:10.532249 11729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:10.532280 11733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:10.532510 11720 server_base.cc:1034] running on GCE node
I20260812 03:52:10.532781 11720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:10.533099 11720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:10.534220 11720 hybrid_clock.cc:648] HybridClock initialized: now 1786506730534206 us; error 32 us; skew 500 ppm
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:10.536746 11720 init.cc:377] Logged in from keytab as kudu/127.7.132.189@KRBTEST.COM (short username kudu)
I20260812 03:52:10.537186 11720 webserver.cc:460] Webserver started at http://127.7.132.189:38479/ using document root <none> and password file <none>
I20260812 03:52:10.537427 11720 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:10.537487 11720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:10.537632 11720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:10.538618 11720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/data/instance:
uuid: "ed058f718e114794ab00de39867744f0"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "57c68addf48bf1f1fa1e7b8d86f5e301"
server_key_iv: "f1c3c19335dd7a350a88785ce3b39e9c"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.539091 11720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/wal/instance:
uuid: "ed058f718e114794ab00de39867744f0"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "57c68addf48bf1f1fa1e7b8d86f5e301"
server_key_iv: "f1c3c19335dd7a350a88785ce3b39e9c"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.540541 11720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:52:10.541277 11743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:10.541445 11720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:10.541427 11707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } attempt: 2
I20260812 03:52:10.541549 11720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/wal
uuid: "ed058f718e114794ab00de39867744f0"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "57c68addf48bf1f1fa1e7b8d86f5e301"
server_key_iv: "f1c3c19335dd7a350a88785ce3b39e9c"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.541635 11720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:10.541771 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.541658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42066)
0812 03:52:10.541705 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.541735 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 03:52:10.542181 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.542061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42082)
0812 03:52:10.542109 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.542145 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 03:52:10.542287 11707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.189:36419: Network error: Client connection negotiation failed: client connection to 127.7.132.189:36419: connect: Connection refused (error 111)
I20260812 03:52:10.560665 11720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:10.562789 11720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:10.562942 11720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:10.566896 11720 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.189:36419
I20260812 03:52:10.566943 11821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.189:36419 every 8 connection(s)
I20260812 03:52:10.567332 11720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/data/info.pb
I20260812 03:52:10.568567 11824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:10.570989 11824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:10.573494 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 11720
I20260812 03:52:10.573602 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-1/wal/instance
I20260812 03:52:10.573823 7698 external_mini_cluster.cc:1428] Setting key 7deca0f7dea1dbdbd03451a7acdfc92b
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
I20260812 03:52:10.582255 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.571773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59856 (local address 127.7.132.190:42811)
0812 03:52:10.571848 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.571850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.572187 (+ 337us) server_negotiation.cc:407] Connection header received
0812 03:52:10.572781 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.572782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.572796 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.572852 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.573769 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.574565 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.575978 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.576166 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.576258 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.578556 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.578557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.578558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.578567 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.580621 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.581154 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.581155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.581156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.581220 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.581517 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.581518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.581518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.581667 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.581743 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.582061 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.582151 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:52:10.582468 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.571747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42811 (local address 127.0.0.1:59856)
0812 03:52:10.571923 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.571982 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.572669 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.572925 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.572929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.573354 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.573637 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.573643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.574675 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.574676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.575895 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.575897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.576284 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.576286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.576407 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.576889 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.576897 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.578456 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.580730 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.580738 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.580740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.581071 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.581302 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.581303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.581304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.581441 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.581742 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.581748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.581907 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.582022 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.582193 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260812 03:52:10.583272 11824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.132.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.132.188@KRBTEST.COM" created.
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:52:10.592211 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.583583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42096)
0812 03:52:10.583780 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.583788 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.583824 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.583970 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.583970 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.584094 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.584138 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.584139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.585486 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.585486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.585967 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.585969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.586508 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.586510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.586599 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.587037 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.587043 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.588634 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.590836 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.590837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.590837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.591076 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.591326 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.591327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.591328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.591401 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.591803 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.591804 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.591857 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.591930 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.592058 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260812 03:52:10.592314 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.583654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42096 (local address 127.7.132.189:36419)
0812 03:52:10.583812 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.583816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.583840 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:52:10.583865 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.583868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.583882 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.583969 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.584200 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.585401 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.586037 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.586416 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.586495 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.588738 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.588747 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.588747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.588754 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.590736 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.591171 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.591174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.591174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.591246 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.591513 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.591514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.591514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.591714 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.591806 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.591992 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.592131 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260812 03:52:10.593089 11824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:10.593467 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.593317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58162)
0812 03:52:10.593391 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.593437 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 03:52:10.593582 11746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
I20260812 03:52:10.594017 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.593930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58176)
0812 03:52:10.593967 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.593989 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 03:52:10.594141 11824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:43053: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
Entry for principal kudu/127.7.132.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab.
I20260812 03:52:10.602265 7698 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWjyozu/build/release/bin/kudu
/tmp/dist-test-taskWjyozu/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:43053
--webserver_interface=127.7.132.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.132.148:42335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.132.190:42811,127.7.132.189:36419,127.7.132.188:43053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:52:10.633455 11707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } attempt: 3
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:52:10.640367 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.633859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42098)
0812 03:52:10.633918 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.633934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.633998 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.634149 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.634151 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.634319 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.634384 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.634386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.635159 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.635160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.635663 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.635665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.636046 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.636047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.636118 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.636580 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.636586 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.637926 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.639523 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.639523 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.639524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.639665 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.639856 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.639857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.639857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.639927 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.640167 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.640168 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.640200 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.640246 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.640308 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 03:52:10.640419 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.633934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.7.132.189:36419)
0812 03:52:10.634002 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.634004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.634036 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:52:10.634071 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.634072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.634086 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.634143 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.634471 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.635078 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.635747 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.635977 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.636049 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.638003 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.638005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.638005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.638012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.639454 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.639725 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.639726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.639727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.639784 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.639996 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.639997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.639997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.640110 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.640171 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.640274 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.640341 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 03:52:10.640779 11707 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:10.641090 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.640989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58186)
0812 03:52:10.641026 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.641050 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 03:52:10.641419 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.641364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58198)
0812 03:52:10.641387 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.641407 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 03:52:10.641510 11707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:43053: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
I20260812 03:52:10.643349 11824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } attempt: 1
I20260812 03:52:10.643654 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.643559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58206)
0812 03:52:10.643604 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.643634 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 03:52:10.643927 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.643861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58222)
0812 03:52:10.643886 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.643907 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260812 03:52:10.644014 11824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:43053: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
I20260812 03:52:10.673199 11707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } attempt: 1
I20260812 03:52:10.673615 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.673503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58224)
0812 03:52:10.673554 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.673581 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 03:52:10.673858 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.673797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58238)
0812 03:52:10.673821 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.673837 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260812 03:52:10.673914 11707 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:43053: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
W20260812 03:52:10.698203 11840 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:52:10.698457 11840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:52:10.698494 11840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:52:10.698532 11840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:52:10.699831 11840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:52:10.699900 11840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:52:10.699931 11840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:52:10.699963 11840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:52:10.699995 11840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:52:10.701557 11840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.132.148:42335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.132.190:42811,127.7.132.189:36419,127.7.132.188:43053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/krb5kdc/kudu_127.7.132.188.keytab
--principal=kudu/127.7.132.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.132.188:43053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.132.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 02:28:51 UTC on ccycloud.quasar-yovhtb.root.comops.site
build id 4581824
I20260812 03:52:10.701853 11840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:52:10.702119 11840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:52:10.706905 11848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:52:10.706908 11849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:10.706970 11840 server_base.cc:1034] running on GCE node
W20260812 03:52:10.707078 11852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:52:10.707293 11840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:52:10.707576 11840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:52:10.708734 11840 hybrid_clock.cc:648] HybridClock initialized: now 1786506730708735 us; error 41 us; skew 500 ppm
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:52:10.711212 11840 init.cc:377] Logged in from keytab as kudu/127.7.132.188@KRBTEST.COM (short username kudu)
I20260812 03:52:10.711687 11840 webserver.cc:460] Webserver started at http://127.7.132.188:34705/ using document root <none> and password file <none>
I20260812 03:52:10.711918 11840 fs_manager.cc:359] Metadata directory not provided
I20260812 03:52:10.711983 11840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:52:10.712138 11840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:52:10.712128 11824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } attempt: 2
I20260812 03:52:10.712529 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.712403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58248)
0812 03:52:10.712472 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.712504 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 03:52:10.712852 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.712763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58262)
0812 03:52:10.712805 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.712833 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 03:52:10.712936 11824 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.132.188:43053: Network error: Client connection negotiation failed: client connection to 127.7.132.188:43053: connect: Connection refused (error 111)
I20260812 03:52:10.713414 11840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/data/instance:
uuid: "d05bb8732d994335bba890571b75cc14"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "bfd4cf27024a7ca31322601112945b4b"
server_key_iv: "3e1617189a2578bdf9c6412765ed0f12"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.713780 11840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/wal/instance:
uuid: "d05bb8732d994335bba890571b75cc14"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "bfd4cf27024a7ca31322601112945b4b"
server_key_iv: "3e1617189a2578bdf9c6412765ed0f12"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.715206 11840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 03:52:10.715909 11867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:52:10.716074 11840 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 03:52:10.716174 11840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/data,/tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/wal
uuid: "d05bb8732d994335bba890571b75cc14"
format_stamp: "Formatted at 2026-08-12 03:52:10 on dist-test-slave-39l8"
server_key: "bfd4cf27024a7ca31322601112945b4b"
server_key_iv: "3e1617189a2578bdf9c6412765ed0f12"
server_key_version: "encryptionkey@0"
I20260812 03:52:10.716244 11840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:52:10.728377 11840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:52:10.730073 11840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:52:10.730219 11840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:52:10.734351 11840 rpc_server.cc:307] RPC server started. Bound to: 127.7.132.188:43053
I20260812 03:52:10.734395 11946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.132.188:43053 every 8 connection(s)
I20260812 03:52:10.734791 11840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/data/info.pb
I20260812 03:52:10.735796 11947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:52:10.737294 11947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:10.737861 7698 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWjyozu/build/release/bin/kudu as pid 11840
I20260812 03:52:10.737984 7698 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWjyozu/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786506457253237-7698-0/minicluster-data/master-2/wal/instance
I20260812 03:52:10.738196 7698 external_mini_cluster.cc:1428] Setting key 95fee50d2860568939084a3b38be7161
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:52:10.746658 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.737983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.7.132.190:42811)
0812 03:52:10.738046 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.738048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.738275 (+ 227us) server_negotiation.cc:407] Connection header received
0812 03:52:10.739064 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.739066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.739079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.739144 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.739745 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.740242 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.741042 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.741228 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.741304 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.743526 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.743527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.743528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.743535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.745214 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.745679 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.745681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.745681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.745743 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.746006 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.746007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.746007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.746136 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.746222 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.746520 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.746596 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 03:52:10.746897 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.737954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.190:42811 (local address 127.0.0.1:59858)
0812 03:52:10.738141 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.738180 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.738949 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.739170 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.739174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.739428 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.739633 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.739638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.740311 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.740312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.740957 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.740959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.741336 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.741338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.741446 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.741916 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.741927 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.743436 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.745286 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.745292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.745294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.745603 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.745811 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.745812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.745812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.745953 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.746227 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.746233 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.746388 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.746472 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.746628 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260812 03:52:10.747432 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.738654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.7.132.188:43053)
0812 03:52:10.738793 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.738798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.738856 (+ 58us) server_negotiation.cc:407] Connection header received
0812 03:52:10.738967 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.738972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.738989 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.739084 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.739812 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.740594 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.741276 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.741549 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.741628 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.744312 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.744319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.744319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.744325 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.745888 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.746248 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.746251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.746251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.746308 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.746525 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.746526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.746526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.746673 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.746874 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.747290 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.747365 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260812 03:52:10.747419 11947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:10.749048 11707 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } attempt: 2
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.190@KRBTEST.COM
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.190@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:52:10.755491 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.747798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42114 (local address 127.7.132.189:36419)
0812 03:52:10.747852 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.747855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.747887 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:52:10.747958 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.747959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.747970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.748022 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.748424 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.748960 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.750028 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.750310 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.750373 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.751980 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.751981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.751981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.751989 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.754090 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.754541 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.754542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.754543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.754603 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.754920 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.754921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.754921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.755059 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.755149 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.755310 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.755386 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:52:10.755491 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.747727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.189:36419 (local address 127.0.0.1:42114)
0812 03:52:10.747847 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.747857 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.747900 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.748172 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.748173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.748317 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.748351 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.748353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.749063 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.749064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.749953 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.749955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.750378 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.750379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.750438 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.750807 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.750811 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.751870 (+ 1059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.754175 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.754176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.754176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.754452 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.754729 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.754730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.754731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.754803 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.755137 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.755138 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.755185 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.755263 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.755392 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 03:52:10.756139 11947 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:52:10.756217 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.748201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.7.132.190:42811)
0812 03:52:10.748233 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.748235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.748241 (+ 6us) server_negotiation.cc:407] Connection header received
0812 03:52:10.748259 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.748259 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.748267 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.748314 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.748739 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.749372 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.750189 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.750392 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.750442 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.752976 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.752977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.752978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.752985 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.754963 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.755412 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.755413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.755414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.755469 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.755710 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.755711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.755712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.755818 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.755973 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.756006 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.756118 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 03:52:10.757028 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.749272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58280)
0812 03:52:10.749320 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.749330 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.749368 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.749552 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.749553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.749679 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.749706 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.749707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.750399 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.750400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.751104 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.751107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.751628 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.751629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.751692 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.752250 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.752255 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.753669 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.755996 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.755997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.755997 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.756230 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.756439 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.756440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.756441 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.756505 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.756737 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.756738 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.756777 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.756856 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.756944 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 03:52:10.757058 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.749342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.7.132.188:43053)
0812 03:52:10.749376 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.749378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.749387 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:52:10.749406 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.749407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.749417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.749530 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.749770 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.750303 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.751202 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.751536 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.751824 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.753758 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.753760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.753760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.753767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.755901 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.756316 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.756316 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.756317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.756366 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.756574 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.756574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.756575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.756673 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.756742 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.756878 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.756985 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 03:52:10.758500 11707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2: Bootstrap starting.
I20260812 03:52:10.759227 11707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:10.759549 11707 log.cc:824] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.188@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:52:10.761204 11707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2: No bootstrap required, opened a new log
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.132.189@KRBTEST.COM
I20260812 03:52:10.763365 11707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } }
I20260812 03:52:10.763494 11707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92c99617e9154477af1ee6403c45d5f2, State: Initialized, Role: FOLLOWER
I20260812 03:52:10.763609 11707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } }
I20260812 03:52:10.763994 11707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:10.763960 11962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.764622 11962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:10.764997 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.757024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.7.132.189:36419)
0812 03:52:10.757059 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.757060 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.757140 (+ 80us) server_negotiation.cc:407] Connection header received
0812 03:52:10.757160 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.757161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.757169 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.757208 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.757748 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.758310 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.759200 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.759472 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.759551 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.761680 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.761681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.761682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.761687 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.763140 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.763587 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.763588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.763589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.763646 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.764155 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.764156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.764156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.764311 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.764477 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.764831 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.764907 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 03:52:10.765784 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.756351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58290)
0812 03:52:10.756477 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.756488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.756530 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.756887 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.756888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.757167 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.757201 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.757203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.757975 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.757976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.758511 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.758514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.758985 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.758986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.759036 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.759482 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.759487 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.760510 (+ 1023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.764756 (+ 4246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.764756 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.764757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.764956 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.765155 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.765155 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.765155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.765209 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.765507 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.765508 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.765541 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.765597 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.765690 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 03:52:10.765906 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.756429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58290 (local address 127.7.132.188:43053)
0812 03:52:10.756747 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.756749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.756756 (+ 7us) server_negotiation.cc:407] Connection header received
0812 03:52:10.756774 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.756774 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.756784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.756843 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.757268 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.757905 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.758610 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.758911 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.758971 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.760585 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.760586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.760587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.760593 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.764673 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.765036 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.765037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.765037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.765086 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.765282 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.765283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.765283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.765419 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.765484 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.765698 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.765781 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":87,"threads_started":1}
W20260812 03:52:10.766455 11977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:10.766552 11977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:10.767401 11947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14: Bootstrap starting.
I20260812 03:52:10.768030 11947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:10.768306 11947 log.cc:824] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:10.769037 11947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14: No bootstrap required, opened a new log
I20260812 03:52:10.767107 11593 master_runner.cc:186] Error getting master registration for 127.7.132.189:36419: OK, instance_id { permanent_uuid: "ed058f718e114794ab00de39867744f0" instance_seqno: 1786506730565833 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:52:10.769351 11593 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:52:10.769505 7698 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:52:10.770430 11661 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "92c99617e9154477af1ee6403c45d5f2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59864
I20260812 03:52:10.770545 11661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:52:10.770594 11661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:10.770613 11947 raft_consensus.cc:348] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } }
I20260812 03:52:10.770772 11947 raft_consensus.cc:729] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d05bb8732d994335bba890571b75cc14, State: Initialized, Role: FOLLOWER
I20260812 03:52:10.770834 11947 consensus_queue.cc:260] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } }
I20260812 03:52:10.771072 11947 sys_catalog.cc:564] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:10.771027 11983 sys_catalog.cc:455] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.771126 11983 sys_catalog.cc:458] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:10.771385 11661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } }
I20260812 03:52:10.771590 11661 leader_election.cc:288] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [CANDIDATE]: Term 1 election: Requested vote from peers ed058f718e114794ab00de39867744f0 (127.7.132.189:36419), d05bb8732d994335bba890571b75cc14 (127.7.132.188:43053)
I20260812 03:52:10.772125 11911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92c99617e9154477af1ee6403c45d5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d05bb8732d994335bba890571b75cc14"
I20260812 03:52:10.772255 11911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:10.773438 11911 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92c99617e9154477af1ee6403c45d5f2 in term 1.
I20260812 03:52:10.773658 11626 leader_election.cc:302] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92c99617e9154477af1ee6403c45d5f2, d05bb8732d994335bba890571b75cc14; no voters:
I20260812 03:52:10.773813 11962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:52:10.773962 11962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [term 1 LEADER]: Becoming Leader. State: Replica: 92c99617e9154477af1ee6403c45d5f2, State: Running, Role: LEADER
I20260812 03:52:10.774051 11962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } }
I20260812 03:52:10.774591 11962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c99617e9154477af1ee6403c45d5f2. Latest consensus state: current_term: 1 leader_uuid: "92c99617e9154477af1ee6403c45d5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.774672 11962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:10.776268 11840 master_runner.cc:186] Error getting master registration for 127.7.132.189:36419: OK, instance_id { permanent_uuid: "ed058f718e114794ab00de39867744f0" instance_seqno: 1786506730565833 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:52:10.778393 11840 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:52:10.781431 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.773078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58300 (local address 127.7.132.188:43053)
0812 03:52:10.773111 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.773113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.773883 (+ 770us) server_negotiation.cc:407] Connection header received
0812 03:52:10.773983 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.773984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.773993 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.774021 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.774763 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.775392 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.776280 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.776559 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.776622 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.777974 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.777975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.777975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.777981 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.779704 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.780668 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.780669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.780669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.780738 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.780919 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.780919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.780920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.781021 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.781094 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.781281 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.781352 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 03:52:10.779342 12002 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:10.781716 12002 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:10.782863 12017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:52:10.783087 12017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:52:10.784547 11911 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [term 1 FOLLOWER]: Refusing update from remote peer 92c99617e9154477af1ee6403c45d5f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:10.784847 12015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:10.786002 11983 sys_catalog.cc:455] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c99617e9154477af1ee6403c45d5f2. Latest consensus state: current_term: 1 leader_uuid: "92c99617e9154477af1ee6403c45d5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.786083 11983 sys_catalog.cc:458] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:10.786773 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.773110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.7.132.190:42811)
0812 03:52:10.773147 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.773149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.776325 (+ 3176us) server_negotiation.cc:407] Connection header received
0812 03:52:10.776411 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.776412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.776420 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.776445 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.777106 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.777704 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.778564 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.778897 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.778956 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.781432 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.781433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.781433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.781439 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.783741 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.784169 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.784170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.784171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.784226 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.786210 (+ 1984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.786212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.786212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.786335 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.786401 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.786602 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.786685 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:52:10.788893 11962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92c99617e9154477af1ee6403c45d5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.788976 11962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:10.789067 11983 sys_catalog.cc:455] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92c99617e9154477af1ee6403c45d5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.789168 11983 sys_catalog.cc:458] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:10.789273 12023 mvcc.cc:205] Tried to move back new op lower bound from 7317531569291169792 to 7317531569251831808. Current Snapshot: MvccSnapshot[applied={T|T < 7317531569291169792}]
I20260812 03:52:10.790755 12024 mvcc.cc:205] Tried to move back new op lower bound from 7317531569291169792 to 7317531569251831808. Current Snapshot: MvccSnapshot[applied={T|T < 7317531569291169792}]
I20260812 03:52:10.793738 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.773265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.7.132.189:36419)
0812 03:52:10.773298 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.773300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.778691 (+ 5391us) server_negotiation.cc:407] Connection header received
0812 03:52:10.780130 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.780131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.780141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.780181 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.782164 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.782837 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.787690 (+ 4853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.787983 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.788042 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.790623 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.790624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.790625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.790631 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.792360 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.792870 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.792871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.792871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.792946 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.793168 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.793169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.793169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.793305 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.793402 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.793576 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.793649 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 03:52:10.795035 12017 catalog_manager.cc:1347] Generated new cluster ID: 4f46bbe766b648778a5c9140b657e14e
I20260812 03:52:10.795086 12017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:52:10.802656 12017 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:52:10.803525 12017 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:52:10.804062 11824 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } attempt: 3
Aug 12 03:52:10 dist-test-slave-39l8 krb5kdc[11568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786506730, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.132.189@KRBTEST.COM for kudu/127.7.132.188@KRBTEST.COM
I20260812 03:52:10.812343 11827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.804332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.132.188:43053 (local address 127.0.0.1:58314)
0812 03:52:10.804373 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 03:52:10.804383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:52:10.804423 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:52:10.804582 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:52:10.804582 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:52:10.804737 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:52:10.804775 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.804777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.805645 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.805647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.806416 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:52:10.806418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.806848 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.806861 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:52:10.806918 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.807287 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:52:10.807291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:52:10.808661 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:52:10.810524 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.810525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.810525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.810760 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.811666 (+ 906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:52:10.811667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:52:10.811668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:52:10.811738 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:52:10.812048 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:52:10.812049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:52:10.812104 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:52:10.812165 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:52:10.812252 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 03:52:10.812911 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:10.804423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.7.132.188:43053)
0812 03:52:10.804465 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.804467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.804475 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:10.804495 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.804496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.804504 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.804561 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.804839 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.805548 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.806504 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:10.806788 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:10.806858 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:10.808753 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:52:10.808754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:52:10.808755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:52:10.808761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:52:10.810423 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.811497 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.811498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.811498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.811580 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:52:10.811814 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:52:10.811815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:52:10.811816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:52:10.811941 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:52:10.811996 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:10.812731 (+ 735us) server_negotiation.cc:299] Negotiation successful
0812 03:52:10.812823 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 03:52:10.813026 12017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2: Generated new TSK 0
I20260812 03:52:10.813189 12017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:52:10.814303 11824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0: Bootstrap starting.
I20260812 03:52:10.815099 11824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0: Neither blocks nor log segments found. Creating new log.
I20260812 03:52:10.815398 11824 log.cc:824] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:52:10.823408 11824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0: No bootstrap required, opened a new log
I20260812 03:52:10.825399 11824 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } }
I20260812 03:52:10.825534 11824 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed058f718e114794ab00de39867744f0, State: Initialized, Role: FOLLOWER
I20260812 03:52:10.825608 11824 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } }
I20260812 03:52:10.825919 11824 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:52:10.826020 11781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:52:10.826745 11782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92c99617e9154477af1ee6403c45d5f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ed058f718e114794ab00de39867744f0"
I20260812 03:52:10.826901 11781 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [term 1 FOLLOWER]: Refusing update from remote peer 92c99617e9154477af1ee6403c45d5f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:52:10.827214 12015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:52:10.828243 11782 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92c99617e9154477af1ee6403c45d5f2 in term 1.
W20260812 03:52:10.830456 12052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:52:10.830503 12052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:52:10.830868 12034 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92c99617e9154477af1ee6403c45d5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.830946 12034 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:10.830986 12034 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c99617e9154477af1ee6403c45d5f2. Latest consensus state: current_term: 1 leader_uuid: "92c99617e9154477af1ee6403c45d5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.831029 12034 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:10.831053 12034 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92c99617e9154477af1ee6403c45d5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:10.831086 12034 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:52:10.839339 12056 mvcc.cc:205] Tried to move back new op lower bound from 7317531569405313024 to 7317531569251831808. Current Snapshot: MvccSnapshot[applied={T|T < 7317531569291169792}]
W20260812 03:52:10.849948 11714 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.848196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59894 (local address 127.7.132.190:42811)
0812 03:52:10.848236 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.848238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.849032 (+ 794us) server_negotiation.cc:407] Connection header received
0812 03:52:10.849346 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.849347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.849358 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.849386 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.849889 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59894: BlockingRecv error: recv got EOF from 127.0.0.1:59894 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 03:52:10.850507 11959 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.848408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.7.132.188:43053)
0812 03:52:10.848445 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.848446 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.849122 (+ 676us) server_negotiation.cc:407] Connection header received
0812 03:52:10.849443 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.849445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.849454 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.849480 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.850456 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58330: BlockingRecv error: recv got EOF from 127.0.0.1:58330 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 03:52:10.851157 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.848587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42132 (local address 127.7.132.189:36419)
0812 03:52:10.848620 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.848622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.849258 (+ 636us) server_negotiation.cc:407] Connection header received
0812 03:52:10.850795 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.850796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.850805 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.850830 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.851112 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42132: BlockingRecv error: recv got EOF from 127.0.0.1:42132 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 03:52:10.853497 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.852829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42140 (local address 127.7.132.189:36419)
0812 03:52:10.852859 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.852861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.853042 (+ 181us) server_negotiation.cc:407] Connection header received
0812 03:52:10.853141 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.853142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.853158 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.853186 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.853457 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42140: BlockingRecv error: recv got EOF from 127.0.0.1:42140 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 03:52:10.859690 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.859033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.7.132.189:36419)
0812 03:52:10.859061 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.859062 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.859259 (+ 197us) server_negotiation.cc:407] Connection header received
0812 03:52:10.859337 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.859338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.859347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.859371 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.859658 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42146: BlockingRecv error: recv got EOF from 127.0.0.1:42146 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260812 03:52:10.863497 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.862693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42160 (local address 127.7.132.189:36419)
0812 03:52:10.862738 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.862740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.863015 (+ 275us) server_negotiation.cc:407] Connection header received
0812 03:52:10.863115 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.863116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.863125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.863155 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.863459 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42160: BlockingRecv error: recv got EOF from 127.0.0.1:42160 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 03:52:10.869344 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.868652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42176 (local address 127.7.132.189:36419)
0812 03:52:10.868682 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.868683 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.868909 (+ 226us) server_negotiation.cc:407] Connection header received
0812 03:52:10.868995 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.868996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.869004 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.869030 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.869311 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42176: BlockingRecv error: recv got EOF from 127.0.0.1:42176 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 03:52:10.879168 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.878394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42192 (local address 127.7.132.189:36419)
0812 03:52:10.878435 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.878436 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.878654 (+ 218us) server_negotiation.cc:407] Connection header received
0812 03:52:10.878791 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.878792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.878801 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.878828 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.879122 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42192: BlockingRecv error: recv got EOF from 127.0.0.1:42192 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 03:52:10.897022 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.896239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.7.132.189:36419)
0812 03:52:10.896273 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.896275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.896495 (+ 220us) server_negotiation.cc:407] Connection header received
0812 03:52:10.896597 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.896598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.896607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.896634 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.896942 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42198: BlockingRecv error: recv got EOF from 127.0.0.1:42198 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 03:52:10.931355 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.930447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.7.132.189:36419)
0812 03:52:10.930496 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.930498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.930821 (+ 323us) server_negotiation.cc:407] Connection header received
0812 03:52:10.930896 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.930897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.930910 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.930961 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.931290 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42202: BlockingRecv error: recv got EOF from 127.0.0.1:42202 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 03:52:10.997232 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:10.996478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42214 (local address 127.7.132.189:36419)
0812 03:52:10.996550 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:10.996552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:10.996681 (+ 129us) server_negotiation.cc:407] Connection header received
0812 03:52:10.996766 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:10.996767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:10.996778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:10.996845 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:10.997160 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42214: BlockingRecv error: recv got EOF from 127.0.0.1:42214 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260812 03:52:11.127192 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:11.126398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.7.132.189:36419)
0812 03:52:11.126486 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:11.126487 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:11.126539 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:52:11.126643 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:11.126644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:11.126657 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:11.126752 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:11.127127 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42218: BlockingRecv error: recv got EOF from 127.0.0.1:42218 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 03:52:11.385098 11833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:52:11.384363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.7.132.189:36419)
0812 03:52:11.384432 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:11.384434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:11.384476 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:52:11.384594 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:11.384595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:11.384607 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:11.384673 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:52:11.385003 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39756: BlockingRecv error: recv got EOF from 127.0.0.1:39756 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:52:11.441946 12171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92c99617e9154477af1ee6403c45d5f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92c99617e9154477af1ee6403c45d5f2" member_type: VOTER last_known_addr { host: "127.7.132.190" port: 42811 } } peers { permanent_uuid: "ed058f718e114794ab00de39867744f0" member_type: VOTER last_known_addr { host: "127.7.132.189" port: 36419 } } peers { permanent_uuid: "d05bb8732d994335bba890571b75cc14" member_type: VOTER last_known_addr { host: "127.7.132.188" port: 43053 } } }
I20260812 03:52:11.442090 12171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c99617e9154477af1ee6403c45d5f2 [sys.catalog]: This master's current role is: LEADER
I20260812 03:52:11.782275 12002 catalog_manager.cc:1259] Loaded cluster ID: 4f46bbe766b648778a5c9140b657e14e
I20260812 03:52:11.782346 12002 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14: loading cluster ID for follower catalog manager: success
I20260812 03:52:11.783110 12002 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14: acquiring CA information for follower catalog manager: success
I20260812 03:52:11.783351 12002 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d05bb8732d994335bba890571b75cc14: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:52:11.830925 12052 catalog_manager.cc:1259] Loaded cluster ID: 4f46bbe766b648778a5c9140b657e14e
I20260812 03:52:11.830991 12052 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0: loading cluster ID for follower catalog manager: success
I20260812 03:52:11.831678 12052 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0: acquiring CA information for follower catalog manager: success
I20260812 03:52:11.831931 12052 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ed058f718e114794ab00de39867744f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:52:11.900336 12181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:11.898128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.7.132.189:36419)
0812 03:52:11.898255 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:11.898257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:11.898270 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:52:11.898347 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:11.898348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:11.898397 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:11.898457 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:11.898585 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.899047 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.899709 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.899880 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.899925 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:11.900065 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:11.900167 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:11.900221 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:11.900257 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 03:52:11.900274 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260812 03:52:11.904397 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:11.901976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43004 (local address 127.7.132.188:43053)
0812 03:52:11.902082 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:11.902084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:11.902092 (+ 8us) server_negotiation.cc:407] Connection header received
0812 03:52:11.902153 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:11.902154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:11.902203 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:11.902279 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:11.902368 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.902867 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.903577 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.903796 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.903867 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:11.904023 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:11.904125 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:11.904198 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:11.904281 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 03:52:11.904300 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":51,"threads_started":1}
I20260812 03:52:11.908432 12192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:11.905904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.7.132.188:43053)
0812 03:52:11.905943 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:11.905944 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:11.906016 (+ 72us) server_negotiation.cc:407] Connection header received
0812 03:52:11.906076 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:11.906077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:11.906087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:11.906132 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:11.906293 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.906940 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.907700 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.907928 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.907988 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:11.908095 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:11.908157 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:11.908204 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:11.908242 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 03:52:11.908264 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 03:52:11.908842 12181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:11.905739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.7.132.189:36419)
0812 03:52:11.905776 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:11.905778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:11.906035 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:52:11.906122 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:11.906124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:11.906132 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:11.906158 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:11.906345 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.906853 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.907877 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.908205 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.908268 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:11.908404 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:11.908435 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:11.908484 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:11.908748 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:52:11.908773 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 03:52:11.909260 12205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:52:11.905737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50664 (local address 127.7.132.190:42811)
0812 03:52:11.905853 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:52:11.905855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 03:52:11.905904 (+ 49us) server_negotiation.cc:407] Connection header received
0812 03:52:11.905986 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:52:11.905987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:52:11.906035 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:52:11.906104 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:52:11.906204 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.906935 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.908307 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:52:11.908620 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:52:11.908713 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:52:11.908905 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:52:11.909006 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:52:11.909061 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:52:11.909138 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:52:11.909169 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
I20260812 03:52:11.910852 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 11593
I20260812 03:52:11.926363 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 11720
I20260812 03:52:11.936235 7698 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWjyozu/build/release/bin/kudu with pid 11840
2026-08-12T03:52:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2922 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5831 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274681 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST